Building the CartaX ATS with MarketGrid technology
With the strategic acquisition of MarketGrid Systems, Carta Capital Markets began building the platform that would become CartaX.
Private markets are breaking records. There's more “dry powder”—money ready and waiting to be invested—in venture capital and private equity than there ever has been before. As this capital accumulates, investor demand for private company stock continues to rise. In 2019, Carta created a new subsidiary called Carta Capital Markets, LLC (CCMX) to facilitate trading in private company shares.
Unlike the public stock market, the market for private stock is limited to people recognized as “accredited investors," who typically have more investing experience. This restriction is meant as a protection, because private companies aren't subject to the same reporting requirements as publicly traded companies.
But it also poses a problem: Unless private companies organize a secondary liquidity event, it’s difficult for employees to sell. Carta tasked CCMX with designing a way to increase liquidity in private markets. The goal was to design new transaction types that companies at various stages of development could use to hold secondary liquidity events, which help employees and early investors unlock the value of the equity they hold.
Enter MarketGrid Systems
In February 2020, Carta acquired MarketGrid Systems Pty Limited, a small and innovative company based in Sydney, Australia. MarketGrid specialized in building exchange trading systems (ETS), which are platforms for trading specific commodities or equities. These systems trade a large range of products, including equities, bills, bonds, currencies, commodities, futures, and options. In the U.S., private equities can trade on a subset of ETS known as Alternative Trading Systems (ATS). MarketGrid Systems’s three founders collectively hold more than 100 years of experience in building trading systems for financial exchanges all over the world. The company’s core trading system product, MarketGrid, became the foundation for the CartaX ATS.
What is MarketGrid?
MarketGrid is a third-generation product that improved upon first- and second-generation trading platforms built and operated in the 1980s and 1990s. To provide new and enhanced capabilities, the developer team included new features to improve performance through lower latency and higher reliability.
MarketGrid Systems was in the process of deploying MarketGrid into new markets—including carbon credits, diamond exchanges, and cryptocurrencies—when it caught the attention of the leadership at CCMX. MarketGrid’s unique third-generation features made it the ideal base for the sort of innovative standalone platform that CCMX wanted to develop for trading equity in private companies.
MarketGrid was able to offer CCMX the foundational software infrastructure required to operate an ATS. This infrastructure includes a number of components, processes, and technologies:
- Core Matching Engine (C++)
- Trading and admin GUI and backend (Node.js, TypeScript, C++)
- An internal communications grid (ZeroMQ)
- A native FIX server (C++)
- Independent market database (real-time and historical) (kdb+/Q)
- Tools for developers
- Tools for operations
- A number of APIs and access methods, including:
- Native MarketGrid API (MGAPI) over ZeroMQ
- JSON over Kafka
From a technical perspective, the MarketGrid components and processes are illustrated below.
The core functionality of any ATS is the matching engine, which handles incoming orders, order books, and trade generation. A key feature of MarketGrid is that it does not use a database for this core functionality. Instead, MarketGrid is built around a set of custom internally shared-memory structures that are designed to provide the high performance and the low latency required for financial markets.
The internally shared-memory structures, as well as the code associated with managing and accessing them, are generated automatically during the build process and are not handwritten.
Fully-logged event-driven architecture
MarketGrid employs an event-driven architecture with features that provide high performance and a high level of resilience against all types of failures, thereby reducing the chances of lost orders, inaccurate trades, system outages and delays.
- The internally shared-memory structures that comprise the matching engine may only be updated or written by incoming events.
- The design of the internally shared-memory structures allows for lockless read access by processes other than the matching engine (such as the UI server component).
- All incoming events to the MarketGrid matching engine come via one of the MarketGrid APIs and are sequenced by a logging thread.
- Incoming transactions are logged to an immutable transaction log file that allows for complete replay of the system to any previous state.
- MarketGrid provides tools to examine internally shared-memory and transaction log files in real time.
- The logging thread also handles replication of transactions to remote instances of the matching engine. This provides resilience through both hot and warm backup nodes.
- MarketGrid provides checkpointing using cache files that are special cases of transaction log files. This facilitates persistence, garbage collection, and the fast restarting of the system.
When we originally designed MarketGrid, one of our goals was to employ code generation to have as much of the system generated as part of the build process as possible. This was particularly important for infrastructure-related code, which refers to all code that is not directly related to a particular piece of business logic or functionality, such as database access, internally shared-memory handling, or API and message handling.
Infrastructure code is the foundation on which business logic is written and implemented: It is the base on which the rest of the platform is built. But while the foundation of a building is a more or less permanent structure, developers can update infrastructure code to implement new features.
A few key reasons motivated our goal of maximizing the generation of infrastructure code in the build process:
- Many years of experience working in financial markets and building automated trading systems taught us that over time, large, complex systems like trading systems become increasingly difficult to maintain and enhance, and increasingly less reliable. Much of this is due to increased complexity and inconsistencies in the infrastructure-related code on which the system is built. These are introduced as the system is enhanced and changed to meet changing business requirements.
- Since MarketGrid was designed as a system that can be deployed to a number of different customers—such as financial exchanges, banks, brokers—a key requirement was that it be easy to customize, maintain, and deploy different instances of the system for different customers and requirements. Employing code-generation for core infrastructure makes it much easier to maintain multiple instances of the system than would otherwise be the case.
- By nature, infrastructure code tends to be highly repetitive and predictable, which makes it easy to generate and tedious to write by hand. Generating this code automatically as part of the build process allows developers to concentrate on delivering business functionality, which increases productivity and reduces the surface area for bugs.
- MarketGrid was originally developed by a very small team of three developers. Our goal was to allow the company to grow while keeping the team small and efficient. By building a custom code-generation tool-set first, the developers were able to do this work once, then reap the benefits of ongoing, high-quality, and highly consistent generated infrastructure code on which the business logic could be built and maintained by a very small team.
How code generation works
Code generation in MarketGrid is based on taking a core data model, API definitions, and some other associated data (collectively known as the “data model”), and then using those to generate the core infrastructure code for all components of the system.
MarketGrid’s developers designed a bespoke code generation framework to build all generated code. Source for the data model is written in the TypeScript language, which provides all of the benefits of a strongly typed language, such as early bug detection, when editing and updating the data model.
When the system requires a change—for example, a new field to a table for a specific business requirement—the data model allows the change to be made once, in one place. Then, when the trading system is built, the change will be propagated to all components where it is required. This means that developers won’t have to implement the change in multiple places manually.
As illustrated in the following diagram, the data model generates the shared memory model, core C++ code, the shared memory interfaces, the UI back end (shown below as “nq”), persistence, migration, load and unload code, tools (such as internally shared memory, log file inspectors, and other inspection tools), a test framework, and documentation.
Data model example: sector table
The following code snippet shows how a table (in this case, the sector table) is defined in the MarketGrid data model. It is represented as TypeScript code.
Note that the definition of the fields in the table (in this case ShortName, Name, and Industry) define both a type and a number of decorators (of which there are many), such as .required (which is used to generate code that enforces a value for a required field) and comment. Comments are used during code generation to build the documentation for the system, which is presented as an automatically generated wiki.
All of the code necessary to deal with sectors, in all components in the system, is generated from this data model definition. Maintaining the definition in the data model is much simpler than maintaining the generated code that would otherwise have to be hand-written.
A fully deployed instance of MarketGrid comprises approximately 746,000 lines of source code across a number of languages, the main languages being C++ and TypeScript. For compiled languages, source code is not distributed but used in the build process. However, the number of handwritten lines of source code in MarketGrid is approximately 169,000. This means that 77% of the source code in the system is generated automatically during the build process.
Facilitating iteration in the financial markets
MarketGrid has been an ideal platform on which to build CartaX. In particular, the matching engine and associated infrastructure provide a high-performance platform with the reliability, resilience, and transparency that is required for a highly regulated trading platform.
MarketGrid’s code-generation framework provides the system with greater flexibility, maintainability, and reliability. Efficient code generation has also allowed the CCMX team to design and prototype unique and sometimes very complex trading models, which can then be rapidly built and implemented with a high level of confidence.
See all posts or choose a post below.
The Carta liquidity report: Q1 2022According to Carta data, the market for secondary liquidity is still heating up. Q1 2022 saw a 57% increase over Q1 2021 in the number of secondary deals.
How H1 partnered with CartaX to offer liquidity through a tender offerH1 partnered with CartaX to run a third-party tender offer to satisfy excess investor demand and offer its employees the benefit of realizing some of the value they have built together.
How to prepare for a tender offerCartaX works with private companies to help them plan secondary liquidity transactions. Read below to see the key considerations to think about before designing a tender offer.