When addressing an operating environment such as Telco, the goal must be to engineer a platform that can cater to the explosion in digital services and rich mobile experiences that followed the huge success of the iPhone.

The amount of data traffic on telco networks continues to increase at an exponential pace year on year. And the need for IT systems to process all network traffic in real-time is imperative as Telcos need to use all that data and extract insights to successfully engage with customers using data-driven decisions.

But when you start digging into where the bottlenecks are within traditional Telco applications, the process quickly becomes a substantial excavation. Is the problem to do with application design and business logic execution?

>See also: The rise and evolution of the vertical database

Are things held up by replication and clustering technologies? Well, yes, and you have to solve both of those issues as well, but our findings took us deep into the database persistence layer itself. It turned out that the online transaction processing principles that have underpinned relational and other transactional database designs for the last 50 years were now the root cause of the fundamental performance problems that we needed to solve. The culprit? Database locking.

As discussed earlier in this series, relational databases have no assumptions about the applications that are built on them: they are general purpose, flexible, and invaluable in that jack-of-all-trades role. But relational databases deal with transactional compliance by locking the affected data until each transaction is committed.

Setting up and removing these locks creates significant overhead for a telco network where hundreds of millions of transactions are completed each hour. Because these transactions are spread over a large set of customer data (think: AT&T has more than 100M customers), the actual probability of two transactions colliding is quite low.

As a result, incurring all this overhead for every transaction fails the “engineering 80:20 rule” – make the common case efficient and let the uncommon case pay the price. It doesn’t matter if the database is disk-based or in-memory, SQL or NoSQL, relational or object-based; if the system is guaranteeing integrity by locking all related data for every transaction, then in this vertical, it will deliver unpredictable latency and fail to scale.

>See also: Graph database tech is helping improve businesses: from telcos to Airbnb

This was the realisation that started our technology journey towards a whole new paradigm for database transaction management. To provide a solution, a database for Telco needs to be designed to consistently deliver all of the following characteristics all the time:

Low, predictable latency. The objects in-memory must be available to the business logic application using direct memory access – there can be no locking, no serialisation/deserialisation, no network hops and no complex transformations.

When the business logic reads an object there’s a very small “consistency test” to ensure the object isn’t “half updated” and then memory speed access is granted to the data.

Contention Free. The objects in-memory must be updated by the business logic using a non-blocking, contention-free algorithm. In a nutshell, the business logic can perform updates to the database as part of a transaction without any of the overhead of traditional locking or blocking, but still maintaining strict consistency guarantees.

This makes the database hundreds to thousands of times more efficient for managing write-intense workloads like Telco network event processing and keeps the performance of each transaction consistent independent of other load on the system.

>See also: Telco databases are tricky

Complex processing without impacting performance. Complex network event processing operations must be able to access many objects in the database – the subscriber, the device, many balances, the products owned, the pricing information on those products, a group hierarchy, a hierarchy of balances and purchased products, etc.

A contention free database aligned with a new approach to executing business logic creates a system that is remarkably immune to increasing complexity within the subscriber data model or pricing rules.

The value of moving to a vertical specific database technology lies in the ability to address the scalability needs, subscriber experience and business innovation made possible by a model of data-driven, real-time customer engagement.

In summary, Telco’s require a unique database technology – one that is capable of managing magnitude increases in network traffic driven by everything from customer interactions to IoT and smart devices.

Sourced from Dave Labuda, founder, CEO and CTO of MATRIXX Software