Once upon a time they were the undisputed sheriffs of the IT infrastructure, but Database Administrators (DBAs) are fast becoming less and less popular, particularly in today’s world of 'shadow IT.' Some compare the agile development model and web-based applications to the Wild West with data stores full of structured and unstructured data springing up all over the enterprise.
The danger is that developers working outside DBA controls are creating a new generation of siloes and overly complex legacy systems. This is a challenge. For one, Gartner estimates that by 2017 50% of data stored in NoSQL DBMSs will be damaging to the business due to a lack of applied information governance policies and programs.
> See also: Shadow IT is a reality, but it doesn't have to keep your organisation in the dark ages
And woe is the DBA who suggests caution or attempts to add processes, which slow down the rush to build applications. To stretch the cowboy analogy even further, the DBA is almost like the unwelcome stranger walking into the saloon. Everyone stops, and hides what they’re doing.
Perhaps I’m exaggerating slightly, because relations between application developers and DBAs haven’t deteriorated that far, but attempting to restore order to IT’s Shadowlands is a tough challenge.
Process vs. agility
DBAs have always been responsible for maintaining data flow, stability and integrity around an organisation. As the hype surrounding Hadoop and NoSQL reaches fever pitch there is a sense that IT departments have sacrificed workflow processes simply to demonstrate to the business how responsive and agile they can be.
While NoSQL-only solutions may be ideal for some workloads, their deployment requires some planning and management.
For example, NoSQL-only databases are not ACID compliant. Some claim ACID compliance in a single document, but achieving the robust data integrity that enterprises typically require across their data set requires developing complex applications to do the heavy lifting that a relational database already handles.
NoSQL-only solutions also only store data. They don’t process it. Data must be brought to the application for analysis. The application (and hence each individual application developer) is responsible for efficiently accessing data, implementing business rules, and for data consistency.
This is complicated, because each NoSQL database product uses a different representation for its data and its data access/manipulation language. So, organisations may find themselves with multiple but incompatible NoSQL solutions.
NoSQL-only solutions cannot support stored procedures, and NoSQL solutions do not represent any one single technology, inhibiting the ability to reuse code, establish standards and find talented resources. Ultimately, enterprises using NoSQL- only solutions are battling a proliferation of data silos because each application requires its own data store and enterprise data becomes fragmented and loosely governed.
Postgres advances in NoSQL
This is creating potential downstream problems that could have serious consequences. In the UK we’ve seen what happens when regulators respond to significant IT systems failures.
Luckily, in Postgres, there is a technical solution, which goes a long way to addressing the problem. For instance, you can combine unstructured data with relational tables, all the while maintaining ACID compliance and centralised business processing rules and logic.
With JSON/JSONB for supporting and processing document data and the HStore data type for key/value pairs, Postgres provides the flexible data models that developers need to build applications capable of evolving with changing business objectives.
Postgres also allows developers to utilise JSON oriented document syntax directly inside SQL statements, complete with a large supply of functions for manipulating JSON data and converting it back and forth with relational data. The ability to create unstructured data stores and combine components with relational tables on the fly is a powerful capability—unheard of in a relational database.
Postgres supports unstructured data stores but then enables developers to apply schema rules to selected data according to business needs. With Foreign Data Wrappers (FDWs), developers can integrate external structured and unstructured data within Postgres and enable Postgres to read and write SQL queries to foreign data sources. There are FDWs for MongoDB, CouchDB, MySQL, Redis, Neo4j and even Twitter and more.
> See also: The top five data admin errors costing companies millions – and how to fix them
In conclusion I will also stress one point. Integrating this world of shadow IT is not just a technical one, even though I believe the innovations in Postgres go a long way to avoiding the complexity and silos that might otherwise be created.
But we also must avoid this latest stage in the evolution of the enterprise IT environment becoming the 'Gunfight at the OK Corral.' Both application development and database administrator teams must work together to ensure IT is seen as a key source of competitive advantage and supports business strategy and Postgres will go a long way toward ensuring that everyone gets along.
Sourced from Pierre Fricke, VP Product, EnterpriseDB
