Local models will soon be mainstream on consumer devices

Anusheel BhushanAnusheel Bhushanco-founder of PlankOriginally posted on LinkedIn 

In the small models vs cloud models debate, we're overlooking the fact that consumer hardware devices are still only at the cusp being ready for running these models locally. Even Apple MacBooks, phones are not quite there yet. But my guess is that within a year, we will reach that point. And yes we will always live in a hybrid world of local and cloud models, but local models will become a mainstream part of consumer device operating systems.

  • Models and research

Keep reading

Software architecturePlank

Why Plank relies on humans for system design

Here at Plank we believe in relying on humans for coming up with product plans, system designs and core architecture. Being able to bring together business, product and engineering perspectives and getting to the gist of things is something AI still needs more training on. Plans created by AI are often over-engineered, generic and miss out on the magic often. It is ok to provide the detailed plan to AI and let it explore some options or propose language alternatives to communicate. We foresee this to be an increasingly rare skill going forward. An early example of this is Ernst & Young awarding $100 million in bonuses to workers who demonstrate human skills, such as leadership, judgment, business acumen, collaboration and an ability to adapt. An easy way to make sure you don't lose your way is keeping an invariant that you always need to have a block diagram and information flow of whatever system you are building in your mind.

Enterprise AIPlank

The SaaS business model is unraveling in enterprise HR

At Plank, we are seeing first hand the unraveling of the SaaS business model. Large F500 as well as mid-market customers are coming to us to move away from expensive SaaS HR solutions that are rigid, built in a pre-AI world and very slow to work with for changes. In our experience, we are able to build completely custom solutions that are deployed in customer's cloud, with fully local and private LLM under the hood and with full IP owned by them; all in a matter of weeks. So they are comparing solutions that are 8-figure annual costs with a multi-year implementation with just a 6-week implementation and very low one-time implementation costs with close to zero on-going maintenance costs. Also the solution is built in an AI-native way using the latest approaches. Since these techniques were just developed in the last 6-months, existing incumbents simply are not architected in this way.

Coding agentsModels and research

A 3-layer model architecture for agentic coding workloads

In our experiments, we found that using open source models like Qwen or Deepseek via OpenRouter looks temping and cheap on a per call or $/tokens. However, if you run a real benchmark with your pull requests, these models end up taking more rounds before they converge when running agentic workloads. The way to go is to deploy your own open source models. The architecture we go with is a 3-layer one. The 1st layer is local models running on developer machine, the 2nd layer is open source models deployed in our own cloud servers and the 3rd layer is the latest models offered by the labs. We also are able to distribute the workload based on the type - eq a pull request review can run on a slower machine as its not sync.

Software architecture

DBOSify replaces Temporal Python with plain Postgres

Really cool work from Peter Kraft and Qian Li, PhD - DBOSify is a drop-in replacement for Temporal Python that uses Postgres instead of a Temporal server. This lets you run durable workflows, activities, signals, updates, retries, and recovery without needing any infrastructure except Postgres. The other project that took a similar approach was Absurd (github.com/earendil-works/absurd). Excited to try DBosify, hoping there's a Typescript version soon as well. DBOSify GitHub: github.com/dbos-inc/dbosify-py

All postsOlder posts