Bots Don't Care About Money

If you're a non-technical founder or PM vibing your way to the next disruptor, you might be feeding your profits to the furnace. In my own project, using React, NextJS, Clerk, Tanstack, and Pusher, among other things, I was working on a simple notification system. User A interacts with User B's content, which sends a notification to B. Basic stuff. I spent over an hour writing up the rules and requirements to give the AI a very clear illustration of what to build and how it should work. The first pass was very promising, so I began the process of fine-tuning. Occasionally the counts would be out of sync, showing only 4 new notifications when there were 5. No big deal, this happens all the time in React. At this point the feature was nearly complete, about 90% functional, just this one last piece of peculiar bugginess. The bot was struggling to correct it though, and after numerous failed attempts, I started really digging deep into the code myself, line by line, where before I had allowed with a cursory glance. I literally gasped out loud (and added some color commentary to boot). The AI was using React useEffect like duct tape on a hand-me-down Goldberg machine. This part is a little technical... This method will cause a component to update (re-render) whenever there's a change in state, like a notification count. I had given it direction to create a Context Provider, which is a chunk of code that manages a complex piece of state across multiple components; since they all share context, each will update or re-render as needed to keep its own UI in sync. So in this example the NotificationBell, a simple button on the user sidebar component, shows the count using the same data as the NotificationList, which is a completely separate standalone component for displaying notifications, marking them as read, etc. Click on the bell to open the list, mark a notification as read to update the count on the bell, yeah? The bot built the Provider inside a useEffect (for you React engineers, you read that right: >200 lines of code declared as a single const, and the entire provider, with all its Tanstack caching, socket connections, etc, all being recreated on every render. I missed it at first since the useEffect was near the top of the file, and the dependency array looked like a context value. For you non-engineers, this breaks common convention so egregiously that I didn't even look for it. You would never build a provider that way, nor even think to check that it wasn't.) Additionally, it was managing state binding through other useEffects. As part of it, in order to keep the state "clean", it would unsubscribe from a Pusher channel and then promptly resubscribe a few lines later in the same function. In short, this resulted in exponential (potentially millions) of redundant calls to the database, Clerk, and Pusher. All of which charge for usage. Instead of using built-in caching provided by Tanstack, instead of using Pusher sockets to handle channel subscription, instead of using useState, useCallback, useMemo or any number of far more efficient methods to manage state, it had scratch-built a Gordian Knot of useEffects that would have ultimately cost a staggering amount of money in extraneous third-party integration calls. Were I not an experienced engineer, I likely would not have caught it until my user base had grown large enough for it to be evident, by which point the entire system would have needed a rewrite, and I would have already lost a ton of money, and would continue to until the fix was in place. Don't misunderstand me, I'm not saying not to use AI to build your dream project. I am! But if you're not technical, if you're relying on AI to work out nuanced, complex details and get it right, you need to understand that the technology just isn't ready for that yet. To wit: the cover image for this post was generated by AI. It took a dozen or so prompts to get it right, but it looks great, and I was ultimately really happy with the result. Until I counted the fingers and teeth. Did you notice that? I bet not. Do yourself a favor: include an in-depth code review in your project roadmap and budget. If you can't hire an engineer to build the thing, hire one to safeguard it at least. A good engineer will be able to find these kinds of scary hidden bugs, not to mention other really important stuff like security, stability and observability. The last thing you want is to launch your project with a six-fingered bot handling your costs and liabilities. You'll be lucky if all you lose is money.

May 5, 2025 - 18:53

If you're a non-technical founder or PM vibing your way to the next disruptor, you might be feeding your profits to the furnace.

In my own project, using React, NextJS, Clerk, Tanstack, and Pusher, among other things, I was working on a simple notification system. User A interacts with User B's content, which sends a notification to B. Basic stuff.

I spent over an hour writing up the rules and requirements to give the AI a very clear illustration of what to build and how it should work. The first pass was very promising, so I began the process of fine-tuning. Occasionally the counts would be out of sync, showing only 4 new notifications when there were 5. No big deal, this happens all the time in React. At this point the feature was nearly complete, about 90% functional, just this one last piece of peculiar bugginess. The bot was struggling to correct it though, and after numerous failed attempts, I started really digging deep into the code myself, line by line, where before I had allowed with a cursory glance. I literally gasped out loud (and added some color commentary to boot).

The AI was using React useEffect like duct tape on a hand-me-down Goldberg machine.

This part is a little technical...
This method will cause a component to update (re-render) whenever there's a change in state, like a notification count. I had given it direction to create a Context Provider, which is a chunk of code that manages a complex piece of state across multiple components; since they all share context, each will update or re-render as needed to keep its own UI in sync. So in this example the NotificationBell, a simple button on the user sidebar component, shows the count using the same data as the NotificationList, which is a completely separate standalone component for displaying notifications, marking them as read, etc. Click on the bell to open the list, mark a notification as read to update the count on the bell, yeah?

The bot built the Provider inside a useEffect (for you React engineers, you read that right: >200 lines of code declared as a single const, and the entire provider, with all its Tanstack caching, socket connections, etc, all being recreated on every render. I missed it at first since the useEffect was near the top of the file, and the dependency array looked like a context value. For you non-engineers, this breaks common convention so egregiously that I didn't even look for it. You would never build a provider that way, nor even think to check that it wasn't.) Additionally, it was managing state binding through other useEffects. As part of it, in order to keep the state "clean", it would unsubscribe from a Pusher channel and then promptly resubscribe a few lines later in the same function.

In short, this resulted in exponential (potentially millions) of redundant calls to the database, Clerk, and Pusher. All of which charge for usage. Instead of using built-in caching provided by Tanstack, instead of using Pusher sockets to handle channel subscription, instead of using useState, useCallback, useMemo or any number of far more efficient methods to manage state, it had scratch-built a Gordian Knot of useEffects that would have ultimately cost a staggering amount of money in extraneous third-party integration calls.

Were I not an experienced engineer, I likely would not have caught it until my user base had grown large enough for it to be evident, by which point the entire system would have needed a rewrite, and I would have already lost a ton of money, and would continue to until the fix was in place.

Don't misunderstand me, I'm not saying not to use AI to build your dream project. I am! But if you're not technical, if you're relying on AI to work out nuanced, complex details and get it right, you need to understand that the technology just isn't ready for that yet. To wit: the cover image for this post was generated by AI. It took a dozen or so prompts to get it right, but it looks great, and I was ultimately really happy with the result.

Until I counted the fingers and teeth. Did you notice that? I bet not.

Do yourself a favor: include an in-depth code review in your project roadmap and budget. If you can't hire an engineer to build the thing, hire one to safeguard it at least. A good engineer will be able to find these kinds of scary hidden bugs, not to mention other really important stuff like security, stability and observability.

The last thing you want is to launch your project with a six-fingered bot handling your costs and liabilities. You'll be lucky if all you lose is money.