

It also supports a variety of index types which have different speeds and limitations, we discussed this here as well. MySQL has very simple storage engines like a csv engine and very complex ones like InnoDB. Also some storage engines are simple and fast but only work with a small dataset while other storage engines are way more complex but can handle loads of data efficiently.Ī good example for how different storage engines can be within the same database is MySQL which has a variety of storage engines we discuss here before. In particular, there is a big difference between storage engines that are optimized for transactional workloads and those that are optimized for analytics. While this might look like getting into a low level territory of dealing with database internals, application developers who don't work with database internals directly still need to understand the main differences between data structures used in the storage engines of different databases to be able to not just select the suitable database for their application business usecase, but also the suitable storage engine for their database. Why Should I as an Application Developer care about Storage Engines? In later articles we will discuss more topics related to storage and retrieval, mainly, transactions processing and column-oriented storage. This chapter is pretty long so we will discuss it over a series of articles, this one will discuss the data structures commonly used for storage such as hash indexes, SSTables, LSM-trees, B-trees, etc. Now we move on to chapter 3 which discusses storage and retrieval. But you can’t take averages as gospel.In the previous two articles, we discussed chapter 2 of Designing Data Intensive Applications by Martin Kleppmann which discussed data models and query languages, here and here. Recently we performed a PHP upgrade and experienced a 30% improvement in average API response time.

Of course, decreasing your average response time is a good sign. That’s why Amazon looks at the top 0.01% of users and what speeds those high LTV users experience – not the average user. Those are the users you need to test with and benchmark against. High value users (especially!) need fast, reliable responses. An athlete with 6 years of training data, hundreds of sessions performed, and millions of pounds lifted is going to require more data processing and transfer than a new user.īut the question isn’t about relative speed, it’s about requirements and acceptable speed. Coaches and athletes with the most training data also experience the slowest API response times.

We experience this uneven distribution challenge every day at TrainHeroic.

In other words, the customers with the most purchases and highest LTV (life time value). At Amazon, the customers with the slowest page request times are the customers with the most data.
