With the emergence of mass amounts of data, the popularity of the NoSQL database architecture has increased along with its more efficient processing architecture that is commonly associated with big data.
Now, in the instance that you are looking to store and analyse high volumes of that data, younger developers may quickly opt for Open Source NoSQL, claiming it is the database architecture of the future, and some among the Silicon Valley startup culture may even proclaim that the relational database architecture of SQL is no longer relevant.
While on the other hand, seasoned database administrators – and developers with more than a few gray hairs – refer to high-profile examples where NoSQL weaknesses led to the demise of Flexcoin and Poloniex, two Bitcoin exchanges that failed when hackers exploited weaknesses in their respective NoSQL architectures. The more opinionated among these long-time database pros may even proclaim that NoSQL is not suitable for mission critical applications at all.
Not surprisingly, the truth lies between these two extreme views. Both architectures have their place. In fact, both architectures can – and should – be able to coexist.
Rather than play to the extremes, it’s more useful to focus on the specific deployments where applications require accurate data in real-time, particularly in environments with a low tolerance for inconsistent data, such as e-commerce, financial, shipping, transportation and manufacturing. In these examples, eventually consistent data is not an option, and an inconsistent database can kill the business… or worse.
For mission critical deployments, administrators need immediately consistent data, a guarantee that the data will be immediately available and consistent across the entire application, even if it’s distributed. To aide in achieving this level of protection, developers and administrators need databases where every transaction is protected by Atomicity, Consistency, Isolation and Durability (ACID) properties.
Basically, when eventually consistent is not an option, only ACID-compliant databases are sufficient. This means that every database transaction will be processed reliably, in order and consistently, with clear-cut processes to ensure crashes or other events do not affect the quality and consistency of the data.
It’s important to identify and qualify ACID-compliant databases that can deliver the level of reliability, performance and consistency required to support the business.
ACID-compliant databases should have supporting documentation to validate fault tolerance, transaction processing speeds and reliability. Ideally, the database should have a proven history as well, with case studies and references available to validate the claims of the vendor – or the open source architect.
Some of the most popular Open Source NoSQL data stores, such as MongoDB or Cassandra are easy to build, and applications can be quickly coded and deployed – an attractive proposition for entrepreneurs racing to roll out applications in time to generate revenues. Yet it’s also critical to realize that these databases are considered to provide Basically Available, Soft state, Eventual consistency (BASE) semantics, in contrast to ACID-compliant guarantees of high availability.
While popular NoSQL architectures may not deliver ACID-compliant guarantees, some NoSQL implementations can – and do – deliver the immediately consistent ACID properties needed for many mission critical deployments. In fact, one major financial services organisation currently uses an immediately consistent ACID-compliant NoSQL database to process up to hundreds of thousands of financial transactions per second, in a highly regulated market.
It’s also worth noting that even though some businesses, such as online retailers, might require an immediately consistent ACID-compliant database for financial transactions and inventories; they may also be able to safely utilise an eventually consistent BASE-compliant database for managing customer reviews within a social media application, where eventual consistency (BASE) is sufficient.
For those who take an almost religious stance in favor of NoSQL or SQL architectures, it’s also important to consider the fact that an SQL architecture is better-suited for some types of business intelligence analytics and reports, where structured relational analysis is required.
It’s about using the right database to fit the specific application requirement – the right tool for the right job.
So, for mission critical applications that require the ability to scale and handle thousands, or even hundreds of thousands of transactions per second, and that are dealing with data that has to be immediately accurate, developers will want to explore an immediately consistent ACID compliant NoSQL database.
For applications that may not require immediate data consistency, such as a social media community or other environments where eventual consistency can be tolerated, BASE compliant NoSQL database options may work well enough.
For those scenarios where advanced analytics, reporting and other business intelligence analysis are needed, an SQL architecture may be preferred.
Lastly, there are also database options available today that can offer a blend of both NoSQL and SQL technology, over the same data.
The bottom line? With all of the powerful database choices available today, it’s more important than ever to properly identify the requirements for the application, and then choose the right database, or databases for the job. Taking that extra time can mean the difference between a sustainable, productive deployment – and a failed enterprise.
Sourced from Randal Hoff, VP of engineering services, FairCom
