His Rule From High-Frequency Trading for AI Agents
About this episode
I tried building Tabs — a $91M AI billing platform — from scratch with Claude and a cron job. CTO Deepak Bapat walked me through every layer I'd miss: context objects that push contract accuracy from 80% to the high 90s, the commercial graph no horizontal AI can replicate, and why older models sometimes outperform newer ones.
Key moments
- 00:00 What if AI could run your entire finance department?
- 01:08 How a high-frequency trader ended up building AI for finance
- 03:25 What does an AI billing platform actually do?
- 09:15 Are CFOs already using AI agents for reconciliations?
- 11:05 Should you build billing in-house or buy a platform?
- 13:36 What is a commercial graph and why can't horizontal AI build one?
- 17:54 Can AI agents actually replace human finance teams?
- 24:21 Will one person run finance for a $100M company?
- 27:07 How will AI agents from different platforms communicate?
- 28:54 Can you build a $91M AI product with Claude and a cron job?
- 35:23 Is AI accuracy a model problem or a context problem?
- 39:48 Why did this CTO delete the chat interface?
- 45:10 Are developers losing their coding skills because of AI?
- 46:42 Why invest in junior engineers when AI ships faster?
- 48:36 Should coding interviews be replaced with code review?
- 51:16 The book every interdisciplinary builder should read
Full transcript
Early on, we made a bet that we didn't want to build a chat interface. Actually, finance people don't really want that. Just go in and click three buttons and fix it. If customer A has an AP portal and merchant X has signed customer A and then merchant Y goes and signs customer A, we can start to drive insights on how the invoices should look such that they pass the AP portal. Those are the sorts of insights that we can generate. You can't get a standard horizontal AI product. I actually don't think it's just a model problem. I actually do think there is a context problem here.
There are some earlier models that actually do a better job of being able to process contracts with the right context object than newer models with that same context object. There will be a company that grows to 100 million in ARR with one person running finance across all of these various tools. One of our CFO design partners told us, "Look, the whole reason they've hired me is to blame someone when things go wrong." The other day I was playing around with some new libraries. I actually decided I'm going to go write the code myself. And it was surprising how quickly the skills deteriorated. There was a minute where I was just staring at a blank page being like, okay, where do I start?
Today I'm talking with Deepak Bapat. He spent years building high-frequency trading systems where one mistake could cost millions of dollars. And now he's CTO and co-founder of Tabs building AI for finance. Welcome, Deepak.
Thank you for having me.
Do you have a version of yourself that's not on LinkedIn that you can share with us?
Yeah. Yeah, absolutely. So, I graduated with a computer science degree back in 2011 and joined and I've been in I've basically been in New York ever since. After graduating, I joined an HFT company here for about I think it was about two three and a half years. And then quickly pivoted into startups earlier and earlier stage. So, the first startup I went to was a company called Yext, which was out of New York. And that's really where I fell in love with it. I mean, the culture there was incredible. The talent was incredible. Every just like with HF, what was cool is like, you know, back then, it wasn't always the case that you would walk into a standard software development shop.
And the reason I joined a trading company first is because the talent was just so astronomically incredible. And then when I got to Yext, I actually felt the same thing. And that's kind of what made me fall in love with it is I could build systems for large cohorts of people and work with individuals who were just way better at the thing that I wanted to get good at than I was. And so spent a few years there and then kind of took on the opportunity to be really early stage at a company called Latch in New York City.
And we were a vertically integrated hardware software company. I ended up running a significant amount of the software engineering team, distributed systems, doing security, cryptography, talking to hardware devices. And so what's been cool is going between a series of different domains and becoming an expert in a series of different domains. And I and I've now ended up at Tabs where I'm the CTO and co-founder with Ali who's my CEO and Rebecca who's our other co-founder and lead strategy and implementation. And so it's been an incredible ride all kind of I have a big New York City bias. So it's it's been incredible to build here.
Speaking of Tabs, right, can you explain what Tabs does at let's say three levels? One to elementary schooler and one to a finance manager still using spreadsheets like another one to like CFOs deciding whether to trust AI with their revenue.
Yeah. So for the elementary schooler, the way I kind of present it is look, when one company is selling to another company, you basically need to keep track of money, right? How much does your customer owe you? Did you get paid? And Tabs is built for that so that you can worry about building the best product for that company. That's how I'd explain it to, you know, an elementary schooler. And for the finance manager still in spreadsheets, the way I think about it is everything is built off of your contract. And a single contract is the start of a relationship that you have with a customer. And there are then additional not just written down amendments, but additional amendments across the relationship that continue to compound that financial relationship between you and your customer. And Tabs starts out by reading your contract to understand the basis of the relationship. And then builds on top of that such that there are two basically tracks of work that we are focused on. One is making sure that invoicing, billing and payment is incredibly efficient for you.
Minimizing revenue leakage across that track and the other one is making sure that you are able to achieve a completely positive revenue audit the next time you go get audited. For the CFO specifically, the way I would kind of describe it is we are leveraging LLMs, classic machine learning and determinism specifically so that you can run on these two tracks, minimize revenue leakage and like a positive audit opinion while minimizing human capital expenditure as your company scales. So you should be scaling sublinearly from a human capital and resourcing perspective while your company scales exponentially.
Who on the finance team should use Tabs?
Yeah. So it's a comp it depends on the type of company. So there are some companies that we engage with who already have really established finance teams and look historically speaking as companies have scaled you kind of scale your finance team in a like a correlated way. So there are companies where we come in at a point where the team is already quite large and we have controllers using the system, managers using the system, outsourced accountants using the system and the CFO using the system. So that's one cohort. The other cohort is the company that has maybe one operator or one person in finance that is now hitting a sort of growth trajectory where they're thinking about bringing on the first or second real finance person. And in those situations, our goal is to be the most useful we can be for them and for whatever outsourced teams that they're using because most people our ICP is generally SaaS and professional services. And in both of these verticals, there's a lot of offshoring used that we want to try to help minimize or augment.
What we're finding is, you know, there is an accounting first off there's an accountant shortage, right? So there are not if you if you read the literature in the news like there are not enough CPAs and accountants graduating every single year. So that's really part of the reason that we started Tabs. But then the other thing we've realized is like look if you can take money and resources and spend it on the product, spend it on the places that help your business grow like that is the best use of that capital and we want to help you achieve that.
Let's say if you're doing the first time, is there a moment that they will go from skeptical to like I actually need this? Is there a wow moment and what does that look like?
Yeah, it's it's a good question. I think it depends on the team. Look, I think when people most of the people are engaging with us, they actually kind of get it from the start. I mean, contractual relationships and trying to historically trying to handle billing and revenue off of a contract has been the most tedious part of the finance stack and the revenue stack in general. It is the most complex and it is it is often times the work that is in some ways it's it's uncelebrated. And so most of the people that we are talking to and working with like they understand that coming in where they I think they get that aha moment is that first time you drop the you drop a contract into the platform it pulls it out it's providing you know anomaly detection it's providing the invoices pre-generated and then being able to kind of see the path to reducing time spent on these rote jobs of following up on emails right because once they start to see that in the platform. I think they it just kind of clicks cuz they know it's a pain.
Everyone is coming to us knowing that this is the pain. I mean, early on, right? Like you're probably using a QuickBooks or a Xero like general ledger that is this simple but requires a ton of manual work and that works until let's say you have 50 customers and you have to keep track of collections and aging and you have to start doing revenue recognition across usage. Especially today, you're seeing a lot more startups getting into the usage game, right? They're doing upfront prepaid commitments, complex tiered usage, and that's where it actually starts to all break down because now you're building bespoke spreadsheets for every customer, whereas with Tabs, it's it's kind of a one-size-fits-all. Our AI will figure out what the right structure of the of the data should look like so that you get the right invoices and revenue out of the platform.
And I read about a months ago Dell's CFO announced that you know he's using AI agents for like reconciliations and journal entries. Sounds like they could be your customer.
So that's that's kind of what's cool about that is like I think the reconciliation journal entries month-end close is the piece that is most exciting to your CFOs and your controllers because every year they have to go through they have to go through this audit right and especially as you go IPO or your public company that audit becomes so much more complex.
What's kind of hidden behind the surface though are things which I like to think about a lot which is like the collections piece, the billing piece, like how many people are you bringing on to just follow up to make sure you have cash in the bank. And then what does the abstraction look like before the journal entries and the close such that when you look at it, you know it's correct and you don't have to worry about the downstream implications of that month end close because it's all been taken care of via abstractions. And that's kind of what Tabs is built around, right? Like Tabs is at the end of the day, an invoice is an abstraction on top of a journal entry. And Tabs wants to automate the generation and the management of those abstractions.
It's a people managed thing too because you're sending out this invoice and then you're sending to a real human. At least at this point, it's still a people management.
Yeah. Yeah. The way we talk about it is that billing, invoicing and payments is a customer relationship problem as much as it is it is a just work and doing the actual rote like clicking of buttons problem. And this is where I think understanding context understanding who you are sending the invoice to become so powerful.
But for a company like Dell right you may consider building it in house. At what stage should a company consider investing in a product like Tabs?
So when I was at Yext, I helped on the on the billing and subscriptions platform. And I will tell you that which is what I tell most people is if you're working on a system that is not part of the core product, like the number of incredibly talented and top tier engineers that were working on this that should have been focused on the actual product was a substantial suboptimal resource allocation. At the time there was nothing you could do. That was the only thing you could do. This was 12 years ago, but today with Tabs, my take would generally be it is not your core area of expertise. If the goal is to build the best product and Dell ships a ton of great products, but like if you're if the core is to go do that, then go focus on that and spend those resources on that and let us handle the rest of it.
And I think that has to be the way like even internally, right? Like when we think about build versus buy, we're not we don't use our company doesn't need to necessarily use like servers in sitting in an actual data center, right? We've abstracted it away using AWS and GCP and whatnot. And we don't, you know, when we think about deployment, we're not we're not running that on like bare metal. Always be thinking about that build, license, buy model no matter what. So that's one. The second piece that I like to talk about is this idea that most CFOs, most people that are working at building, most controllers, let's say you're changing a job every two to three years, right? You're probably going to see somewhere between 8 to 10 companies in your prime.
Where you'll be able to pattern match by that 10th company across. But what we are able to do is just within just by gathering enough merchants with enough customers, we are now seeing lifetimes worth of data and insights that we can leverage in order to better equip our customers. And I think that is that's really what this whole vertical AI agentic play is about. We have so much enterprise rich enterprise data that the that almost those like the horizontal labs don't get access to because everyone has signed an enterprise term term agreement with the with the frontier labs such that we can actually use that and learn and build a context graph around it.
Can you unpack that a little? What exactly do you know from your database that other people don't know?
Yeah. So I think a really simple one is let's say we have customer A and there's something in finance called AP portals, right? So these are accounts payable portals. You're submitting invoices. You have to submit with various information. Really, you know, when you think about the Fortune 100, Fortune 500 enterprise companies, their AP portals oftentimes are so incredibly complex, it's almost it can take years to just get an invoice paid because you have to submit something and oh, the description on the invoice wasn't exactly what was expected. So, you know, you have to try again and every time you try again, you're basically resetting the amount of time they have to go pay.
Well, what we can start to capture is in this process we can start to understand if customer A has an AP portal and merchant X has signed customer A and then merchant Y goes and signs customer A, we can start to drive insights on how the invoices should look such that they pass the AP portal on customer A. And those are the sorts of that's like a really simple example, but those are the sorts of insights that we can generate. And it starts all the way from the point of processing contract understanding how your contracts align in terms of if you're a usage and commitment based model and you're and you have a you have a customer and we know that customer pays late, maybe we're gonna suggest on your renewal that you should throw in a commitment up front. There's a lot of things that we can now do with this rich data that you can't get from like a standard horizontal AI product.
Is there any privacy concerns for leveraging industry level knowledge that
It's a really great question right and the thing that I say is like the thing that we do is when it comes to anonymization and aggregation we anonymize and aggregate data in such a way that your information will not leak. So the one thing I will call out is like we do use the frontier labs for most of our LLM handling. And so we don't we don't train our own models. We don't build our own models. And so what is nice about that is you don't have to worry about your data leaking because none of your data is actually being trained on for LLM models.
For the classic ML pattern recognition matching we're doing a lot of anonymization and aggregation such that we are working to minimize of yeah the amount of like leakage that can happen but the nice thing about putting in data and getting a number is in like a classic ML case is it's it's a very different type of output than you would get from one of these large language models and so and so there is some how do I describe this there's a little bit buffer that we have there because we're not training our own large language models, but we are very thoughtful about companies that have signed that said you can't do X, you can't do Y. We are very thoughtful about separating out that data.
But your commercial graph is accumulating industry level or merchant business level knowledge which is which is without Tabs, I would have to you know you know basically hit my head on the wall at least several times to pass the let's say the AP portal. But by using Tabs you almost feel like you know what I have a shortcut now because you already know how it works with a specific you know business or company.
Yeah those are the sorts of things that we want to provide to our merchants and part of you joining Tabs is knowing that look we are not exposing who else is who else is selling to this customer or anything like that within the platform. That is not the point, right? But the but the point is that we can leverage how a c how a customer is thinking about paying. What language works well in an email just generally speaking when you are escalated beyond 90 days and we can aggregate that across all all customers like we don't have to focus on one specific customer. We can also do it at a much higher level. Those sorts of insights you're you're just not going to get within one lifetime of understanding.
I mean if I think about for instance the Dell case right they I also heard that they cut a lot of employees like 10% headcounts over the year or so right and in the meantime they're announcing that they're deploying these AI agents yes but of course not all those people are you know getting you know laid off or financy but your opinion how good are these agents or AI agents in general compared with human team members? For finance team.
So I think finance is a special case because there's a ton of liability that plays into anyone that works on a finance team. So I think when you think about there are other roles at a company where leveraging agents on completely unsupervised or leveraging agents and replacing humans with agents is probably okay. I haven't spent enough time thinking about what those roles are, but I'm sure they exist. Specifically in finance though, part of the role of the finance team is to be able to sign your name on a dotted line and say, "Hey, I stand by some set of assertions that is being made with this report that's going to a CPA or to the board or to whatever." it is going to be very like in my opinion our system becomes augmentation that allows for less less headcount growth in the future. But our goal is not to just start having you shed employees today.
Like part of this is how do you become a more savvy operator today versus yesterday and that is really where we want to spend our time. And so I think with finance it's just a slightly different problem because because the like I said the liability expectation on your CFO your controller is substantially higher.
What's the hardest thing a finance team does today?
So we're talking about finance kind of as a monolith but I think there's different groups within the finance team and I think they all have difficult problems to solve. So obviously your CFOs is like their hardest job is setting top level corporatewide strategy like that is not an easy thing. There is so much context that you need to be able to provide or have in order to be able to make those decisions that like I think the idea that you're going to go and just replace a CFO with an agent is in is going to be incredibly difficult.
I actually do think that for controllers I think the hardest thing is actually the part of like setting up the system and then making sure that everything follows that system even as the business is trying to change. So a good example of this is revenue recognition. So what happens with revenue recognition and it's it's a it's a process in which you are basically asserting that your revenue follows a set of rules and policies that have been set out and you are minimizing exceptions and at the end of every year you need to audit all of your past revenue in order to validate that assumption. Very similar to like a SOC 2 compliance revenue compliance is a thing. This is why CPAs do all of these compliance like it's the same style of audit for them. The problem becomes as businesses are changing pricing models, business models, etc. How are you handling things that go from an anomaly or an exception to a fact of business or now standard operating procedure and now having to make sure that these new concepts are being aligned to correctly?
And then, you know, I was I was on a panel yesterday and someone mentioned that especially going into a into an IPO sort of environment where you may be looking to go public or raise at a late stage, every day that you do you do things in a quick dirty don't think about the repercussions, you're just trying to get something done way, it is 3 days on the other side that you have to go spend and fix it. So I think their biggest issue is just making sure that everything is compliant and that is also what Tabs is meant to do. Our job is to help them make sure that revenue which is the largest part of the audit remains a compliant process for them.
It just feels like the level of complexity of for different roles and different teams for finance is just it's just really high and it's hard problem to solve. Are you building for the lowest hanging fruit of all these complexities?
Yes, it's a great question. So, I think early on we did, but we're actually now in a So, taking a step back, I think the first thing is you have to build the primitives and you have to build that trust layer underneath the primitive. So, auditability, compliance are part of your primitive set that you're building. I think early on we were building for the lowest hanging fruit and the nice thing has been as we've scaled the engineering team is we've brought on a series of engineers who really have helped us now build for the more broad use cases. So we really follow this Pareto 80/20. We can now support basically 80 to 90% of use cases and that last 10% we can figure out a way to make it as part of this. I think I think what's what's become very interesting is like when you think about the lowest hanging fruit, right? A lot of that can be solved with like a classic LLM wrapper, right?
So a low-hanging fruit would be like responding to an email of someone who has an escalation, right? Doing that at the shallow level by just passing that into an OpenAI or Anthropic API and saying hey give me an email response to this is in my opinion solving solving in a in a way that gets you quick and dirty but doesn't solve for the for the real value prop. And so building the con like the commercial graph or this knowledge graph that underlays our platform, building the primitives and then building agents that are meant to be orchestrated across long periods of time. That's really what we're building for.
I'm wondering is that last 10% replacing the CFO and if you can you can complete the first 90% are you going to be able to replace the whole finance team and you only need a CFO?
Yeah, it's it's it is a great question. I mean look I think by in the next couple of years there will be a company running on Tabs that we are going to be part of a back office suite. It's very similar to how when you think about HR you've got the Ripplings and the Gustos and the Justworks of the world and you've got for expense and AP you've got Ramp, Navan, Brex, ZipHQ, all these great companies. We are going to be one part of the stack and the ideal situation is that in a couple years there will be a company that grows to a hundred million in ARR with one person running finance across all of these various tools and all these various tools will be talking to each other.
There'll be agents talking to each other amongst them but one person will be able to guide all of finance. That's not going to be every company though depending on the complexity of the company. I do think that, you know, you're going to need you're still going to need some control because at the end of the day, it still comes down to liability. I remember when we first started, one of one of our CFO design partners told us, look, the whole reason they've hired me is to blame someone when things go wrong. And it's the same thing of the accountant. And so you still need that level of someone exists at the company to handle this and is managing all of these various agents just like you know we're not going to get rid of software engineers. I truly don't believe we're just going to get rid of software engineers. There's just going to like the job just changes and I think that's what you're going to see in the stack as well.
What's your vision for a company having a very lean finance team? You know where does Tabs sit in this tool landscape?
Yeah, I think we sit as a as a partner to all of those. And I think what it ends up being is these systems are talking to each other either through something like an agent MCP interface or something similar and the humans are talking. There's always the command console that you can go to and view everything, but for the most part, you're doing everything through a through a an interface where you are just talking to a bunch of agents that are all collaborating across various systems. And I think opening up these systems in a way that allows you to do that I think that is going to be a huge value prop. The thing that I don't want to expose necessarily are the insights directly, but I am okay exposing them via the agent. So the agent can tell you what it's doing and why it's choosing to do it, but the underlying knowledge graph that I have is mine here at Tabs and our agent can execute based on that knowledge graph the same way these other systems will. These agents will be talking to is how I see it playing out.
Do you think Tabs will be the manager for managing is these conversations between agents?
This is an interesting question and it's something I've been thinking about a lot like in DevOps and in the SRE world there's concept of orchestration and then there's a concept of choreography right I my general take is that there will at least in the interim there will still be a human that is the orchestration layer of a series of agents that are able to communicate with each other and is orchestrating that conversation but there will be times where there it is much more choreographed than it is orchestrated and so these agents will be making decisions by cross-pollinating across these systems and then will escalate any decision that needs validation to the human such that human can say yes no change xyz and then they go back to it and I think it's going to be a little bit of a combination of both of these but I don't want to say Tabs is going to be the overall orchestrator it remains to be seen how this plays out but I believe it is going to be more choreographed than there is going to be orchestrates.
I think when you think about choreography, it is by its nature decentralized. You have services reacting to events right amongst each other. But there is but then you still need a shallow layer of orchestration on top to make sure nothing is messing up. But you kind of like these systems will get good enough in the next few years. Tabs will be good enough where you can just kind of let it run and only it's only going to escalate 10% of the of anything to you. And if you're running a 100 million ARR business, that is a huge unlock for you to be able to go do other things. And if you have a couple of systems that are all doing that opens up a significant amount of time for you to go focus on more strategic efforts.
I want to ask about how Tabs build these AI agents reliably. Yeah. But instead of just asking you directly, I want to take a flip the table. Let me try build one of your features from scratch and then you tell me what I'm missing.
All right, let's do it.
Okay. Which feature is your killer feature?
Yes, we can talk about contract processing. I think so
So I'm assuming that a contract gets signed in the in the client interface or portal like Salesforce and then the deal moves to maybe closed-won. So the contract is ready and I can take that PDF now feed it to Claude or use Claude API right and ask to extract the all the information I need like pricing, payment terms, renewal and then Claude can definitely output a structured JSON file and put that in the database. I can even build a cron job that just check the database every day and then I can plug in the Stripe for payment processing or email for collections things like that possibly doable. Yeah. Is my is my setup. But what's not enough? Can you share?
Yeah. So the first thing you have to think about is not every contract is born the same, right? So you have contracts with amendments, contracts with commitments, you have people who are papering documents. So if we sell to a Fortune 500 company, most of the time we're not going to be able to use our paper. We're going to have to use their paper, right? You have new pricing to business models that are constantly coming up. And what you're going to find is that look, if you are a simple customer with a one-page order form with a flat price that you're charging annually upfront, yes, you can feed it to Claude. But as it gets more complicated, you're going to see that these these frontier labs, they're the generic models without the right context are going to start to degrade in performance down into the mid-80s, mid-80% accuracy. And we run accuracy based on like a weighted equation across all the fields that we're trying to calculate against. And so that's the first thing.
So the first thing you actually have to do is understand what type of document is this? Is it an amendment? Does it have commitments? Does it have complex usage? What are the categories that this contract underlines? So that's like part of the pipeline, right? Let's say it's a it's a it's a base level contract with commitments with you know tiered usage etc. What we have done as part of the implementation process. So actually taking a step back. So you're going to notice that first piece of figuring out what type of document it is a combination of taking the embeddings and figuring out against set of samples like which it's closest to. So it's like a comparison problem cosine similarity problem and then also using an LLM to validate that the decision is the wrong. So you're kind of running a couple of different algorithms in the pipeline, right?
Why do you need to do the similarity match? Like what am I comparing with?
So you're comparing with other amendments. It's a classification problem. So you're comparing it amongst a larger corpus of embeddings of other types of documents, whether they are specifically amendments, whether they're order forms, whether they're MSAs, whether they're churn notices, whether they're renewal not like whatever they might be. That's like the first layer. Do it that way because it is much simpler and less costly than actually going to the LLM and trying to feed it to an LLM.
So if we have enough data we can we can actually do that ourselves and then understanding what's in the contract like what are the complexities of the contract you can leverage again you can leverage an LLM with the with the right context being passed in which is part of our as part of our implementation training loop like we generate these context objects during your implementation process with feeding in all of your contracts comparing them against other contracts we have and though like there's a lot that goes on there so you have this context object Now, as you know, what we have found is while basic performance degrades, if you're using the right context, your you know, contract extraction can stay well into the high 90s on accuracy, which to be fair is actually like very good compared to most humans.
So, even when most humans we found are processing contracts of the same company over and over and over again, their ability to be correct consistently also degrades, right? Like they're they're oftentimes missing things or the nice thing about LLMs is they're reading a lot of the context simultaneously whereas like we are only scanning certain parts of the page we might miss net terms that are hidden somewhere etc. So that's the first part of the pipeline. We're using structured outputs to take the unstructured data and turn it into structured data. Then there's a deterministic piece of this which is you have to run generating a series of invoices with the events and whatnot.
Like there are algorithms that you have to run in order to make that work, right? And so it's we've moved away from classic ML and LLMs to now deterministic workflows to generate the invoices, generate your performance obligations, etc. Once those invoices are generated, we're going to send a message out to our merchant over Slack saying, "Hey, this is ready, please go check it or you can just go ahead and auto approve it here." There are some merchants that just go ahead and auto approve, but most people are going in and checking. As they are going in and making adjustments, what we are doing is we actually, we just released this, but we are auto-calibrating your context objects based on changes that you've made in our platform. So that the next time around we are able to capture any new behavioral changes that you want to make against contracts.
As part of that there's a separate flow in all of this which is our anomaly detection which also uses embeddings and tries to use basically uses principal component analysis to try and understand the variety of different buckets of contracts that exist. And when a new contract comes in that doesn't fit any of those buckets, we need to flag it. Have you have you fix it like put it have your a closer eye on it fix it and then we can actually add that to our training set automatically as part of this. So it's like there are yes you can for a very basic contract just get it uploaded get the data stored in a database and like you know if it's the right structure output generate the invoices and that part is fine but this whole like self-correcting piece of the system and the all of the different parts that go into it were things that we had to understand learn and build over
Is this a model problem or is this a context problem what if Claude just get just so much better and you don't even need to do the classification step my you have this classification step which I think increase your accuracy because if I just show the contract to the LLM and then let it you know be burdened of figuring out what contract is and then the like the likelihood of making a mistake is higher then you know a classification problem.
Yes. Yeah. Yeah. I think that's true but I think I think there's a couple thoughts here. So one is there is there are other deterministic workflows that happen down the line where look if you just threw this a Claude and it was an amendment. It needs still does need the context from the previous contract in order to understand what is amended.
Right? So there are context pieces to this but I actually don't think it's just a model problem. I actually do think there is a context problem here. I think there's you know there's been plenty of literature around I think there was something that came out maybe it was in February about how GPT 5.2, the article was like GPT 5.2 can't count to five. It was it was a paper that got published being able to understand the un like being able to give AI the right set of tools that are deterministic to be able to actually generate good output is one problem. When we talk about the context problem, I do think that what we have found in fact is that there are some earlier models that actually do a better job of being able to process contracts with the right context object than newer models with that same context object.
Maybe it is maybe it is an overthinking problem internally. There could be some there could be some element of like a lot of the newer models are somewhat fit to coding problems and very specific types of problems. I just I just generally feel that if you can figure out how to make how to use an a more simple model and use context to drive outcomes that are actually satisfactory, what you are doing is you're doing the best thing for your customer, which is you are reducing co you're actually reducing costs for them while also being able to maintain a level of efficacy. And I think that gets overlooked like I think there's some Nvidia exec that came out recently that was like actually in some cases it's actually cheaper to just hire people than to spend money on compute and we are going to start to see that as these models continue to grow. It's like the models themselves are going to get expensive and so we must be able to pin ourselves to simpler the Haikus, not the Opuses of the world and get those working with our problem.
So I think it is an context engineering problem at the end of the day that we need to solve for if we want to be good fiduciaries and good stewards of our customers.
More reasons to believe that Claude improve improving their models itself is not going to solve this vertical AR problem in finance.
Yes, that's exactly right.
Right. Is hallucination a problem?
Hallucinations can be a problem and we have found so this is an interesting question right because I think you need in some ways you need enough reps for the halluc hall hallucination to actually be to be to show up as a problem. So for instance, if a model makes a mistake one every 10,000 contracts that come in, right, there is some element of most merchants don't have 10,000 contracts. And so most merchants are not going to actually hit this hit this threshold where they are going to they're going to see an error. But that being said, we always build with the assumption that hallucinations or errors are going to be a problem from the models themselves.
When we launch new features, we require that there be a human in the loop on the merchant side that is approving and clicking a button for all intents and purposes that says, "Yes, I am good with this." And so all of our systems until the merchant feels really comfortable, we intentionally toggle on human in the loop. And that's how we've gotten away with it. At the end of the day, we do tell every one of our merchants, the models are only going to be so good. They will be better than a human processing your contracts, but they will not be, you know, 100%. Here's like we actually provide that level of accuracy in the platform itself, so they can see how it's doing.
I'm assuming the finance team using Tabs can still prompt the system directly, right?
There's there the way that we have allowed for that is actually through an MCP server. So early on we made a bet that we didn't want to build a chat interface in the platform. The reason for that is because we found that the background agent model worked better for what our finance teams want. So that bet has kind of paid off because with MCP and with Claude, the Claude harness being as good as it is and the Claude Code harness being as good as it is, it is a much better system in which to interact with Tabs where you need some level of you need some level of stochasticity. And so we have we've kind of put all of our eggs into that basket rather than trying to build an internal chat interface, which we have found to be kind of a nice gimmick, but actually finance people don't really want that.
Like if a we process a contract wrong, there's a way and we're coming out with this shortly. There's a way in which once they make changes, we will learn from that, right? We're going to learn from that and we're going to improve the next time around. But they don't want to go in and like prompt the changes. Just go in and click three buttons and fix it. Like that's the right thing to do because now imagine that prompt like when you prompt it gets it wrong. Now you're already starting to grade real trust. This is why I think the UI experience is still incredibly important and we do focus on that
You're saying you don't have a chat interface and then the way to communicate is through actions.
That's right. And we have focused on if you want to if you want to communicate with Tabs through an through a chat experience use an MCP. And most of what we are doing is actually through what we call background agents. Ambient agents is the LangChain term that they used. But we found that is way more ROI generating than them having to send a request to do something.
Human human communicating with agents actually create an creates another layer of potential hallucination. Do you have suggestions for finance team how to use Tabs?
Yeah, it's it's a really good question. The way we say is like trust that we are building pipelines with the right level of determinism. So even when you go in and you want to re-trigger let's say we've we've retrained the context object and so you want to re-trigger extraction on a given contract. We don't let you put in a prompt. And that is specifically for this reason because we have found that when people try to prompt on top of the context object, they oftentimes cause more issues because now you're giving differentiated advice from what the context object has.
And so these systems get confused and you can actually generate more hallucinations. And so we've actually made it a very clear point that you cannot do that. The place you can prompt in the system is when you are generating new emails to send out as part of dunning and escalations on invoices. You can prompt and say, "Hey, here's a little bit of information about context around XYZ. Please, please add that and that will go into the prompt because you're going to go in and modify that email anyways before you send it." End of day, it goes back to those two things. When you closed-won a contract, you're minimizing revenue leakage and you're getting that you're passing your revenue audit while minimizing additional human capital.
I'm curious, how big is your engineering team?
Yeah, so our engineering team our core engineering team including our AI team is about 30. We have an additional 10 FDEs and then we have you know sales engineers and other people. The core when I when I think about our engineering team, I would say it's it's around 40 to 45 people that we have doing some sort of engineering at the company to drive the product.
Has AI changed how you think about how many people you need and who do you hire?
Yeah, it's this is a really interesting question and something we debate internally a lot. What's what's and I'm sure you have seen this too. It's like just because we have AI doesn't mean people leave earlier. People are still like at the at the office for as late as they were before, but the expectation of what they can ship is much higher. And so the way we see it is if we were the one thing I have started to now do is look at token usage and token costs like we are seeing some like spikes there and we need to figure out how we manage that against a population of engineers right across the spectrum of new to incredibly experienced. But from a pure engineering capacity standpoint, if we have resourced for X, I'm just gonna say we're going to go build more. Like we're not gonna we're not we are less it's almost less limitation on what we can do as a product team.
The other thing it has unlocked for us is actually as you added more people there was a tax on there used to be a tax on communication and that has dramatically decreased because now I can have my product managers look at the codebase directly. I can pull in Notion, Claude, Claude to Notion, to Linear, to everything and understand where everyone's at. And so I think in fact what it allows you to do is it allows you to do more with more do more with less or do the same with less. Like I think being able to do more with more is substantially better.
I do feel there's a type of engineer that's becoming more valuable since AI and that just become less valuable.
The one thing I will caution people is the other day I actually I was playing around with some some new libraries and I in those situations I actually decided I'm going to go write the code myself and it was surprising how quickly the skills deteriorated and I had to like I mean it came back but like there was a minute where I was just staring at a blank page being like okay where do I start? Whereas like if this was a few years ago I would have just started I would have known exactly what to do. And so the one thing I do caution is especially earlier on in your career. I think it's okay to take some time. What I tell some of the more junior engineers is like write the TRDs yourself. Write some of the code yourself. You don't want to just outsource all of your thinking because you will never be able to get it back because you won't have built it in the first place.
That's such a dilemma, you know.
Yes. Yes. Because they want to ship fast and we want them to ship fast. But yeah
Bar is higher, everybody's doing more. And if you're slow trying to figure something out, yeah, do you do you think there's room allow allowing for that on your team? Like junior engineers
It's really difficult, but I'm trying to give them the room and the space because it's only going to make them better if I'm able to allow them to do that, right? Giving them the space now, 3 months from now, if they can just crank like if they just become incredible because they have now a base, like that's that's important. Otherwise, they're just they're never going to get the ability to go beyond where they are, which is actually worse for us. It's that classic like investment problem. When do you invest in them?
It's very rare to hear startups willing to invest in junior team members.
Yeah. I was a junior person at one point and people invested in me, right? So, like the idea that I'm not I don't know. As much as I want to move fast, I want to break things, do all that stuff, I also do believe that at this point in my career, my coaching tree and like the people that are going to go off and do interesting things that have trained under me or whatever is important and the more people that get to go do that and have incredible careers, I think that's important to me.
How are you giving your junior engineers the room for growth?
Yeah, again, this is really difficult. So the first layer is we need to make sure we have exceptional management like the role and I get it like being an incredibly flat company with a lot of agency is important but I actually do think that everyone has blind spots and you need someone who has been there to be able to tell you that things are you know where you are. So I think having a really good is still important. I think the other thing I do is I really really really push them.
So every day I do 20 minutes with some engineer on the team and the goal is to do blockers and whatnot, but I'm also as part of that auditing their TRDs, their work, their GitHub pull requests, and I'm talking to them about it. Like there are certain CEOs and CTO's, exceptional ones that have done this where they're just like they get really nitty-gritty. And so I haven't to be completely honest I haven't found the perfect model in this day and age to grow junior engineers but what I am trying to do is just be more hands-on with them give them really good management and really have them dive in like if you can't explain a PR then you should you need to go back to the drawing board and at least do the bare minimum there and like pushing them to do that.
Are you interviewing engineers differently today?
What's funny, it's actually funny. We switched around from doing a we stopped doing a coding exercise. One of our coding exercise became a code review exercise because we actually think that is almost the more important thing that engineers are able to do today. And so that has replaced one of our on sites, but we still do a coding exercise. We still do a system design because I actually do think that those are important like even for mid-level engineers or junior engineers like you may not understand how to do system design but if we start to prompt you in the right way can you start to think can you adapt like adaptability has become so important that being able to adapt mid-in as you're like getting prompted about different things becomes to me a really important concept to own.
AI do you use on a daily basis and then what do you use AI for yourself?
Yes. Yeah. So I've built so one thing I've done is obviously Claude with MC Claude Desktop and Claude Code is Claude Code is my preferred model although we allow people to use Cursor or Claude here Claude Code whichever they prefer it's probably 50/50. I've also started to build out my own LangChain agents. It's been kind of fun. Like I have an EA now that like listens to my I am by nature not a person to write down to-do lists. And so this thing manages that for me and prompts me whenever I need it. And then the next one I'm I'm I'm starting to build out is this.
It's a tool that helps me think more strategically. First principles, you know, all the different mental models and being able to like come up with the way to answer more tedious questions. So obviously in the day-to-day life with a consumer product, I use ChatGPT and Claude on my phone all the time, but like that is the thing that I am actively building where I'm I'm trying to learn, trying to grow, trying to get better. And it's been it's been it's been great. I actually have like six to-dos that I need to finish by the end of today, which is always the case.
Yeah, a lot of a lot of pressure. I build a MCP server for the founder brains, all the all the founders who I talk to.
Oh, that's interesting.
Yeah, I can I can share once once I make it public, I can share with you as well. I find very very interesting like I asked a very a question that I never asked anybody things like you know if I want to hire this person and what should I let this person do and that's a question I describe the context and then they give me a lot of good feedbacks from founders who I talk to.
Oh that's interesting.
I find it so tremendously helpful. So I will share one one.
That's that's a really cool use of the context that you've g you have with all these transcripts and videos.
If you could recommend a book you know to maybe one of those fresh graduates who are struggling with building their career or maybe yourself today, what would that be?
Yes. My favorite book of all time that I've read is called The Age of Wonder by Richard Holmes. So, it was it came out probably like 10 15 years ago, but I read it in like I must have read in like 2015 or something for the first time. And it's about the romantic era and these great scientists and explorers of the romantic era. And it's so interesting because they're all how do I describe this? They are they're they're not necessarily polymath. So the book is about William Herschel who was like an astronomer and his sister Joseph Banks who like ran the national like the Royal Society for 40 years or whatever talking about how they mixed art and science and like pursued learning and knowledge because they just loved it so much and it was just so fascinating to them.
When you think back at that time like the idea of like a comet right was like a new concept or like exploring Africa understanding like this was the same era of Darwin and all these people and it's like seeing how they and a lot of them were also artists poets those sorts of things and seeing how they merged all of these like interconnected like they were so interdisciplinary and it's it's something that I strive to be I don't think I have I don't think I will ever achieve like multidisciplinary success in that way, but just something to strive for and it's I don't know. It was just like an incredible book. Like wonder is exactly the term I use to describe it.
That's so inspiring. I feel the same way. I think if you're enjoying what you're trying out, then it's you are the wonder you're trying to make yourself into.
Exactly.
Amazing. Right. Yeah. Thank you so much. Thank thanks so much for spending the time and being so generous with your thinking.
No, thank you.
From the TwoSetAI Playbook
What Is Founder Visibility in AI Search?
The definition. Founder visibility in AI search is whether ChatGPT, Perplexity, Google AI Mode, and Claude name and cite a specific founder in buyer-facing answers. How it is measured, and how it differs from PR, SEO, personal branding, and the services sold under the same phrase.
Why Video Content Is Winning AI Search: The GEO Case for Long-Form Video
94% of YouTube citations in a 100-million-citation AI study went to long-form video. See what engines read in a video and how founders can use it.