Why aren't more people/companies using PostgreSQL? Why aren't they moving from proprietary databases to PostgreSQL?
Possible answers: I haven't heard of PostgreSQL. I don't think PostgreSQL will meet my needs. I don't know how to get away from my database vendor.
I'd like to hear some of the reasons for not considering PostgreSQL when creating or migrating a database.

ISVs and PostgreSQL

We should be seeing all ISVs (Independent Software Vendors) migrating their products to PostgreSQL. Why? Because it's one of the fastest ways to increase their ROI and decrease their current customers' costs.

ISVs use a database in their product and typically require their customers to pay for the software license and support. Database software support is an annual expense, so customers have to pay for it, regardless of any special pricing arrangements with the database vendor.

So why aren't we seeing ISVs using PostgreSQL instead of proprietary databases? Some of the reasons, and our responses, follow:

"PostgreSQL wasn't an option when we developed our software."

Maybe PostgreSQL wasn't an option back then; however, if your software is a viable product and you are doing updates, you should plan to include PostgreSQL. Migrating the software to use PostgreSQL - from any of the big proprietary databases such as Oracle or SQL Server - is easier than ever. With tools provided at the PostgreSQL.org site, most migrations can be done in a few days. If your software uses features like Stored Procedures to increase your database efficiency, our tools, available at postgresmigrations.com that are available from Postgres migrations can help migrate those expensive databases to PostgreSQL.

"We allow our customers to choose which proprietary database they want."

Customers should be able to choose between proprietary databases and Open databases. PostgreSQL has the enterprise features that match any of the proprietary databases. Thinking of your customers' bottom line and allowing them to save money on yearly licensing and support costs will enhance your reputation.

"The customer feels more confident in the software when paying for a proprietary database."

Customers will be more confident using PostgreSQL by knowing that they have options for support and that other customers are using it. In addition you can provide current customers with options for how to migrate their current proprietary database to PostgreSQL. This lets your customers know that you are their partner and are looking out for them by helping them to save money. You can provide your customers with potential migration specialists or refer them to the PostgreSQL site.

For a small investment to provide PostgreSQL support in their next future release, ISVs create a significant differentiator between themselves and the competition. ISVs can build a real feeling of partnership with their current customers by providing a lower cost database alternative.

ISVs such as Bowman Systems LLC have already done it, and companies such as VMware have it on their roadmap. What are the rest of the ISVs doing?

Advantages of Migrating from a Proprietary Database to PostgreSQL

The obvious advantage of migrating from a proprietary database to PostgreSQL is the absence of a yearly licensing fee. Regardless of what you currently pay for your licensing, it's cheaper – i.e., zero dollars - to use PostgreSQL. Also, with multiple vendors to choose from, the Maintenance cost can be substantially lower.

Companies that foresee the potential for large savings from this type of migration can consider migrating their databases to PostgreSQL or using PostgreSQL with new projects.

VMware's introduction of the vFabric Data Director makes it possible to save on the Database licensing cost, database operational costs, as well as the reduced infrastructure costs brought by consolidating with VMware.

VMware’s introduction of support for greater than 8 CPUs facilitates virtualization of the largest Database Servers. Very large enterprise users can now move up to extremely large servers, such as the novascale bullion from Bull, which can support up to 160 cores in one SMP system. Their application servers, database servers, mail server and Web Servers can run from one consolidated server, saving on multiple costs (both hardware and software). When PostgreSQL is used exclusively on these massive servers, there are no database licensing issues. That's a striking contrast to the proprietary database vendors who require licenses on all cores in the server.

With this two-pronged approach to Database cost savings, it’s now possible to really leverage your IT dollars to get better performance and substantially lower cost.

When considering a database migration from Oracle to Postgres, one area that requires serious deliberation is that of translating Oracle's PL/SQL source code to Postgres' PL/pgSQL. It is a well documented fact that this translation can very challenging due to the proprietary nature of PL/SQL, and that the translation usually results in expensive recoding and extensive testing of the resulting procedures. It is also well understood that the functionality provided by PL/pgSQL and the semantics of the Postgres server do not necessarily behave in the same manner as the Oracle server and its PL/SQL, even in cases where the syntax is comparable. Going one step further, client code that interfaces with any translated stored database procedure will likely be largely impacted in any conversion. Given these three major issues and a myriad of other details, it is easy to understand that even though the desire is with the Oracle user community to migrate from Oracle to Postgres, the actual implementation will be a very difficult and costly task.

This discussion is largely based on a recent Oracle to Postgres migration that we completed for a large bank in South America. We were contracted to convert a large Oracle Database to PostgreSQL. The complexity of the conversion centered largely on the need to translate approximately 50,000 lines of Oracle PL/SQL. This Stored Procedure code ranged in complexity from simple table interfaces (e.g., INSERT, UPDATE, DELETE and SELECT interfaces) to complex packages using package global arrays. A manual translation of the code was out of the question. The customer, truly following a path of unlicensed open source software, preferred not to use software providing a compatibility layer on top of PostgreSQL. After minimal analysis we determined that the existing tools, including ora2pg, were too simplistic to be of any real use. We did take advantage of the orafce functionality. But, orafce only resolved a fraction of our challenges. We needed something new that would do much more than a line by line translation. We needed a tool that could evaluate the environment and generate code required to support features such as global variables, global package records and global indexed arrays. We needed a tool that could translate complex SQL from one form to another. And we needed utility functions that would build supporting data structures that could provide information to the translator regarding function interface definitions and table definitions.

So we developed a tool and supporting utility functions. Without this tool, the migration of our customer's functions, procedures, packages, and triggers would have been cost prohibitive and practically impossible to code and to test.

The translations done by this tool include:

CONNECT BY OUTER JOIN BULK COLLECT cursors client function interface rewrites including casting as required
Package Global variables
Package Global records
Package Global binary indexed arrays
Exceptions
Postgres SQLSTATE to Oracle SQLCODE translation functionality
Many others

At PGEast, my talk, Large Customers Want PostgreSQL Too !, will include a discussion of this subject.

https://www.postgresqlconference.org/content/large-customers-want-postgresql-too

We had an excellent attendance and a good discussion today at the PGEast Presentation

The presentation covered experiences with large customer database conversions we have recently completed and also described a new tool that was developed to convert Oracle Stored Procedures in the form of PL/SQL to PostgreSQL PL/pgSQL

The presentation is linked:

Large Customers Want PostgreSQL Too !!Oron Slide Rocket

This is one of a series of blogs related to the automated conversion of PL/SQL to PL/pgSQL.

One of the Oracle stored procedure conversion topics that we will be presenting at the PGEast conference in New York is that of CONNECT BY query translation. In this case the issue is mainly the SQL itself. In other blogs I will discuss solutions for more general topics like Oracle Packages and Global Variables.

See our presentation: Large Customers Want PostgreSQL Too !

https://www.postgresqlconference.org/content/large-customers-want-postgresql-too

Linked List Processing

Frequently, the PL/SQL we have translated contained SELECT statements that included the CONNECT BY option. In many cases this feature was used to identify related documents in a document table where the documents were grouped together by a singly-linked list and the CONNECT BY feature was used to traverse the list.

To illustrate this concept consider the following table:

create table songList (
artistId int not null,
id int not null,
priorId int,
title varchar2(20) );

If we assign artistId of 401 to Taylor Swift, then we can map some songs from Taylor Swift’s second album into this table starting with id 201 and begin another list which will ultimately link songs from her next album:

insert into songList values (401, 201,NULL, ‘Fearless’);
insert into songList values (401, 202, 201, ‘Fifteen’);
insert into songList values (401, 203, 202, ‘Love Song’);
insert into songList values (401, 205, 203, ‘White Horse’);
insert into songList values (401, 206, 205, ‘You Belong With Me’);
insert into songList values (401, 301,NULL, ‘unknown’);

The following Oracle CONNECT BY query could be used to list the songs in Taylor Swift’s second album (that is the 200 series):

SELECT artistId, id, title
FROM songList
WHERE artistId = 401
START WITH id = 201
CONNECT BY PRIOR id = priorId;

Which generates a result of:

ARTISTID ID TITLE
========== ====== ====================
401 201 Fearless
401 202 Fifteen
401 203 Love Song
401 205 White Horse
401 206 You Belong With Me

To process the linked list in the same manner using Postgres one must use a Recursive query. My translation tool is capable of translating the prior Oracle CONNECT BY query into a Recursive query automatically. When the prior SELECT query is processed by my translator the following is produced:

WITH RECURSIVE rqName (
artistId, id, title)
AS ( SELECT == the anchor query
artistId,
id,
title
FROM
songList
WHERE
id = 201
UNION ALL
SELECT == the recursive query
tn.artistId,
tn.id,
tn.title
FROM
rqName tp, songList tn
WHERE
tp.id = tn.priorId
)
SELECT artistId, id, title
FROM rqName
WHERE artistId = 401;

The anchor query is responsible for finding the START WITH row. The recursive query then UNIONs its results which are dependent upon the join between the most recent result’s id (tp.id) and the current song’s link field (tn.priorId). The query interates through the list of ids until it encounters the id of 301 which does not have a priorId.

The restriction artistId = 401 could be added to the anchor query. I have found through experimentation that Postgres’ performance is improved when written in the described manner. This improvement, noted by using EXPLAIN ANALYZE is significant when a lot of data is involved.

Tree Hierarchy Processing

The prior recursive query as coded was sufficient to process one hierarchy because of the singly-linked list. A common requirement is that more than one hierarchy, or multiple branches of a tree, must be processed. My translator can make the Recursive query code modifications required to support this for Postgres optionally.

For example, consider the manager / employee relationships in the emp table of the Oracle demo database.

select ename, empno, mgr from emp;

ENAMEEMPNOMGR
==========================
SMITH73697902
ALLEN74997698
WARD75217698
JONES75667839
MARTIN76547698
BLAKE76987839
CLARK77827839
SCOTT77887566
KING7839
TURNER78447698
ADAMS78767788
JAMES79007698
FORD79027566
MILLER79347782

One can use an Oracle CONNECT BY to generate the manager to employee hierarchy:

SELECT ename, empno, mgr FROM emp
START WITH mgr IS NULL
CONNECT BY PRIOR empno = mgr;

ENAMEEMPNOMGR
==========================
KING7839
JONES75667839
SCOTT77887566
ADAMS78767788
FORD79027566
SMITH73697902
BLAKE76987839
ALLEN74997698
WARD75217698
MARTIN76547698
TURNER78447698
JAMES79007698
CLARK77827839
MILLER79347782

The equivalent Recursive query produced from the Oracle CONNECT BY query using my translator with the hierarchy option enabled is:

WITH RECURSIVE rqName (
ename, empno, mgr, arrHierarchy)
AS ( SELECT == the anchor query
ename,
empno,
mgr,
ARRAY[coalesce(empno,0)]== array for perserving the
FROM
emp
WHERE
mgr IS NULL
UNION ALL
SELECT == the recursive query
tn.ename,
tn.empno,
tn.mgr,
arrHierarchy || tn.empno== append to perserve the hierarchy
FROM
rqName tp, emp tn
WHERE
tp.empno = tn.mgr
)
SELECT ename, empno, mgr
FROM rqName
ORDER BY arrHierarchy;

The results for this query are identical to that of the Oracle CONNECT BY.

The key to processing branches of the tree hierarchy is to capture the “path” in the array named arrHierarchy then sort the results based on this path. The path was accumulated in the array using the concat operator in the code:

arrHierarchy || tn.empno

Path Display

The path will be displayed if the Oracle function SYS_CONNECT_BY_PATH is referenced in the SELECT list. The means that the array arrHierarchy will be available and probably should be cast to VARCHAR.

Level Emulation

My translator also can customize the Recursive query to display the equivalent to the Oracle LEVEL pseudo column when the SELECT list includes the Oracle LEVEL keyword. The following is the code generated for the query that evaluates the emp table when LEVEL is selected for Oracle:

SELECT ename, empno, mgr, LEVEL FROM emp
START WITH mgr IS NULL
CONNECT BY PRIOR empno = mgr;

ENAMEEMPNOMGRLEVEL
================================
KING78391
JONES756678392
SCOTT778875663
ADAMS787677884
FORD790275663
SMITH736979024
BLAKE769878392
ALLEN749976983
WARD752176983
MARTIN765476983
TURNER784476983
JAMES790076983
CLARK778278392
MILLER793477823

The translation of the prior query with the LEVEL specification follows:

WITH RECURSIVE rqName (
ename, empno, mgr, LEVEL, arrHierarchy)
AS ( SELECT == the anchor query
ename,
empno,
mgr,
1,
ARRAY[coalesce(empno,0)]== array for perserving the hierarchy order
FROM
emp
WHERE
mgr IS NULL
UNION ALL
SELECT == the recursive query
tn.ename,
tn.empno,
tn.mgr,
tp.LEVEL + 1,
arrHierarchy || tn.empno== append to perserve the hierarchy
FROM
rqName tp, emp tn
WHERE
tp.empno = tn.mgr
)
SELECT ename, empno, mgr, LEVEL
FROM rqName
ORDER BY arrHierarchy;

Conclusion

Rewriting CONNECT BY queries can be a very challenging task to perform by hand. My translation tool can relieve much of the work involved for most of the forms of CONNECT BY that we have encountered. There can be some forms that require special hand recoding such as CONNECT BYs in a subquery (because Postgres does not support recursive queries in a subquery). Nevertheless, the Oracle CONNECT BY needn’t be viewed as an insurmountable hurdle when considering a PL/SQL to PL/pgSQL migration.

Why an open source database is good for business

Annual payments to the DBMS vendor shouldn’t make up the largest portion of your yearly IT expenditures. Your staff is trained, your applications run fine, now is the time to embrace the growing IT business trend and make use of “Open Source” software.

The Enterprise State of Open Source Today

Open source software has progressed significantly from the days when you had to roll-your-own, download it from a website, compile, and hope it worked. There was no real support and the opportunity to run enterprise class applications was largely non-existent.

But those days are gone. Today we have powerful enterprise class tools like: Linux (Red Hat), JAVA, Apache, JBOSS and many others. These products are still available to download; but, they have world-class support companies that stand behind them with the knowledge and the scale to support enterprise offerings at prices that are significantly lower than industry standards for equivalent functionality.

PostgreSQL the Enterprise, Open Source Database

PostgreSQL is “the world’s most advanced open source database” with a feature set that rivals products like Oracle. At Bull we have been managing customer enterprise scale database installations for the past 40 years. We have made a lifetime career of helping Fortune 500 companies integrate their various DBMS systems and convert from one DBMS to the next. For the past 20 years the emphasis has been integrating the data of the enterprise to UNIX then Linux systems: Oracle, Ingres, Informix, DB2, Windows SQL Server and a variety of other DBMS systems. For the past 10 years PostgreSQL has been the new target of integration.

Designed in Silicon Valley (at Cal Berkeley) where Oracle, Sybase, SQL Server and many others originated, Postgres has derived its feature set from 20 years of competition and development with the “Big Three” (Oracle, DB2 and SQL Server). I can tell you that Postgres is just as feature capable as any of the “Big Three” and nearly as performant. Combine it’s open source nature and you have an enterprise scale product that makes a lot of fiscal sense and puts a death sentence on predatory pricing practices.

At Bull, we help major companies convert from their expensive, proprietary databases to powerful open alternatives. I have participated in quite a few conversion projects over the years and have yet to see an open database install that didn’t meet the demands of the project specification on Linux/UNIX, Windows or mainframes.

If you are interested in learning more about Postgres take a look at their website:
www.postgresql.org

If you would like to see the kind of success customers have when Bull migrates them from their proprietary database then take a look here:
Bull modernizes CNAF’s core business applications

When considering a new database system or any new software system that will become a cornerstone of your IT, you have to ask "Who else uses this system and what do they think about the product'. This is especially true of an Open Source product. Because there is no vendor standing to make millions from the sale of the software, you will not see Super Bowl ads touting Free Software and there is no Marketing team providing glossy brochures. So how do you find the other major users of Open Source Software.

In the case of PostgreSQL we have first hand knowledge of customers that we have helped to make the switch to Postgres. But there are other sources that are readily available. Easiest to read about are on line testimonials (http://www.postgresql.org/about/quotesarchive):

Skype, We have been using PostgreSQL as the main DB for most of our business needs right from the start.

City of Garden Grove, I can't say enough good things about PostgreSQL. From backups to application development to administration, PostgreSQL has been a joy to work with.

Some of the more recognizable Postgres Users:

- Sony
- Vonage
- FTD Florists
- Nippon Telephone and Telegraph, Japan's largest telecommunications company

The Postgres community sponsors multiple conferences on the East Coast and West Coast of the U.S.
Some recent presentations:

Skype: runs their entire business on Postgres.

Federal Aviation Authority (FAA): are mapping all US airfields and
their landmarks/beacons etc, using PostGIS on Postgres. Also, they
have/are moving the NOTAM (Notice To Airmen) alerting system to
Postgres. Both systems have obvious reliability requirements.

MyYearbook.com is a very large US based social networking site that
uses Postgres to support millions of users.

Enova Financial: Managed over $1 billion in loans with Postgres last year alone.

Wisconsin Court Systems executes about 80 million database transactions against about 100 databases distributed around the state.

In our experience we have helped two very large organizations move their data and applications to PostgreSQL:

CNAF, One of France's largest social service organizations have moved their largest Batch and OLTP applications to Postgres and run 1 Billion SQL statements per day. http://www.bullservices.com/data-migration-postgresql/postgresql-success-stories/

In Brazil, we have converted the check processing for Brazil's largest banks Caixa Bank and Banco do Brasil from Oracle to Postgres.

So there are many examples to choose from... what's holding your organization back from starting on the Open Source road using PostgreSQL?

Ken.Rosensteel@Bull.com

The ideas, opinions, recommendations and views expressed in this message are strictly my own and in no way commit or represent those of Bull HN as my employer or Bull HN's Internet provider.