Skip to content

2025#

Dear Mark: About Those $100M Signing Bonuses

An open letter regarding Meta's superintelligence talent acquisition strategy

So, Mark, we hear you're in the market for AI talent. Nine-figure signing bonuses, personal office relocations, the works. Well, consider this our application letter.

Your recent offers to top researchers from OpenAI and Anthropic show you understand what's at stake in the race to superintelligence. We've been working on a complementary approach: building infrastructure that gives AI systems the ability to create their own tools, write their own code, and extend their own capabilities in real-time.

Here's the critical insight: Artificial Superintelligence will be achieved when AI systems can improve themselves without human developers in the loop. We're building the concrete infrastructure to achieve that goal. Our platform enables AI to write code, create tools, and enhance its own capabilities autonomously.

Who We Are (And Why You Should Care)

We're engineers working on the frontier of AI capability expansion. We've successfully executed multiple deep learning projects that current LLMs simply cannot do - like training neural networks to read EEG signals and discover disease biomarkers. This experience taught us something critical: LLMs are fundamentally limited by their tools. Give them the ability to build and train neural networks, and suddenly they can solve problems that were previously impossible.

The gap between Llama and GPT-4/Claude isn't just about model size – it's about the surrounding infrastructure. While Meta focuses on training larger models, we're building the tools and systems that could dramatically enhance any model's capabilities. Our System Prompt Composer demonstrates significant improvements in task completion rates for open models. Add our MCP tools and the gap shrinks even further.

Deep Learning Projects That Prove Our Point

We've successfully delivered multiple deep learning projects that current LLMs cannot accomplish:

EEG Signal Analysis: We trained neural networks to read raw EEG data and recognize biomarker patterns with high accuracy. No LLM can do this today. But give an LLM our infrastructure? It could design and train such networks autonomously.

Financial Time Series Prediction: We built models that ingest market data, engineer features like volatility indicators, and train models to predict price movements. Again, ChatGPT can't do this - but with the ability to create and train models, it could.

Medical Image Classification: We developed CNNs for diagnostic imaging that required custom architectures and specialized data augmentation. LLMs can discuss these techniques but can't implement them. Our infrastructure changes that.

These aren't toy problems. They're production systems solving real challenges. And they taught us the key insight: The bottleneck isn't just model intelligence - it's also model capability.

What We've Built

Machine Learning Infrastructure Ready for Autonomy

Here's where our work connects most directly to your superintelligence objectives. We've built sophisticated infrastructure for machine learning that humans currently operate - and we're actively developing the MCP layers that will enable AI systems to use these same tools autonomously:

Component Neural Design: We built a visual canvas where humans can create neural architectures through drag-and-drop components. The key insight: we're now exposing these same capabilities through MCP, so AI agents will be able to programmatically assemble these components to design custom networks for specific problems without human intervention.

Training Pipeline Infrastructure: Our systems currently enable:

  • Experiment configuration and management
  • Distributed training across GPU clusters
  • Real-time convergence monitoring and hyperparameter adjustment
  • Neural architecture search for optimal designs
  • Automated model deployment to production

What we're building now: The MCP interfaces that will let AI systems operate these tools directly - designing experiments, launching training runs, and deploying models based on their own analysis.

The ASI Connection: Once our MCP layer is complete, AI will be able to design, train, and deploy its own neural networks. This creates the foundation for recursive self-improvement – the key to achieving superintelligence.

Why build vs buy? You could acquire similar capabilities from companies like Databricks, but at what cost? 100 billion? And even then, you'd get infrastructure designed for human data scientists, not AI agents. We're building specifically for the future where AI operates these systems autonomously.

MCP Dynamic Tools: Real-Time Capability Extension

Our Model Context Protocol (MCP) implementation doesn't just connect AI to existing tools – it enables AI to create entirely new capabilities on demand:

def invoke(arguments: dict) -> str:
    """AI-generated tool for custom data analysis.

    This tool was created by an LLM to solve a specific problem
    that didn't have an existing solution.
    """
    # Tool implementation generated entirely by AI
    # Validated, tested, and deployed without human intervention

When an AI agent encounters a novel data format, it writes a custom parser. When it needs to interface with an unfamiliar API, it builds the integration. When it identifies a pattern recognition problem, it designs the neural network architecture, writes the training code, executes the training run, evaluates the results, and deploys the model autonomously.

This isn't theoretical - we've built and tested this capability. Each tool creation represents an instance of AI expanding its own capability surface area without human intervention. The tools are discovered dynamically, executed with proper error handling, and become immediately available for future AI sessions.

System Prompt Composer: Precision AI Behavior Engineering

While the industry practices "prompt roulette," we've built systematic infrastructure for AI behavior design. Our System Prompt Composer (written in Rust with native bindings for Python and Node.js) provides software engineering discipline for AI personality and capability specification:

  • Modular Prompt Architecture: Behaviors, domains, and tool-specific instructions are composed dynamically
  • Context-Aware Generation: System prompts adapt based on available MCP tools and task complexity
  • Version Control: Every prompt configuration is tracked and reproducible
  • A/B Testing Infrastructure: Systematic evaluation of different behavioral patterns

This is how we're working to close the Llama-GPT gap: Our enhanced prompting system gives Llama models the contextual intelligence and tool awareness that makes GPT-4 impressive. Early tests show promising results, with significant improvements in task completion when open models are augmented with our infrastructure.

The platform enables rapid iteration on AI behavior patterns with measurable outcomes. Instead of hoping for consistent AI behavior, we engineer it. The composer automatically includes tool-specific guidance when MCP servers are detected, dramatically improving tool usage accuracy.

Execute-as-User: Enterprise Security Done Right

Unlike other MCP implementations that run tools under service accounts or with elevated privileges, LIT Platform tools execute with authentic user context. This security-first approach provides:

# In Docker deployments
subprocess.run(['gosu', username, 'python', tool_path])

# In on-premises deployments  
ssh_client.execute(f'python {tool_path}', user=authenticated_user)
  • True User Identity: Tools execute as the actual authenticated user, not as root or a service account
  • Keycloak Enterprise Integration: Native SSO with Active Directory, LDAP, SAML, OAuth
  • Natural Permission Boundaries: AI tools respect existing filesystem permissions and access controls
  • Complete Audit Trails: Every AI action traceable through standard enterprise logging
  • No Privilege Escalation: No sudo configurations or permission elevation required

This means when an AI creates a tool to access financial data, it can only access files the authenticated user already has permission to read. When it executes system commands, they run with the user's actual privileges. Security through identity, not through hope.

Real-World AI Workflows in Production

We've moved beyond demonstrations to production AI workflows that replace traditional business applications:

Real-World Applications We've Enabled:

  • AI systems that can ingest market data and automatically create trading strategies
  • Code generation systems that don't just write snippets but entire applications
  • Data processing pipelines that adapt to new formats without human intervention
  • Scientific computing workflows that design their own experiments

The key insight: Once AI can create tools and train models, it's no longer limited to what we explicitly programmed. It can tackle novel problems we never anticipated.

Why This Matters for Superintelligence

ASI won't emerge from training ever-larger models. It will emerge when AI systems can develop themselves without human intervention. The path forward isn't just scaling transformer architectures – it's creating AI systems that can:

Self-Extend Through Tool Creation (The Path to Developer AI)

Our MCP implementation provides the infrastructure for AI to discover what tools it needs and build them. When faced with a novel problem, AI doesn't wait for human developers – it creates the solution. This is the first concrete step toward removing humans from the development loop.

Self-Improve Through Recursive Learning (The Acceleration Phase)

Our autonomous ML capabilities let AI systems identify performance bottlenecks and engineer solutions. When AI can improve its own learning algorithms, we enter the exponential phase of intelligence growth. An AI agent can:

  • Analyze its own prediction errors
  • Design targeted improvements
  • Generate training data
  • Retrain components
  • Validate improvements
  • Deploy enhanced versions
  • Critically: Design better versions of itself

Self-Specialize Through Domain Adaptation

Instead of general-purpose systems, AI can become expert in specific domains through focused capability development:

  • Medical AI that creates diagnostic tools
  • Financial AI that builds trading strategies
  • Scientific AI that designs experiments
  • Engineering AI that optimizes systems

Self-Collaborate Through Shared Infrastructure

Our team-based architecture enables AI agents to share capabilities and compound their effectiveness:

  • Tools created by one AI available to all team AIs
  • Knowledge graphs shared across sessions
  • Learned patterns propagated automatically
  • Collective intelligence emergence

Self-Debug Through Systematic Analysis

Our debugging infrastructure applies software engineering discipline to AI behavior:

  • Comprehensive error handling with stack traces
  • Tool execution monitoring
  • Performance profiling
  • Automatic error recovery
  • Self-healing capabilities

The Opportunity Cost of Not Acting

While Meta focuses on model size, competitors are building the infrastructure for AI agents that can:

  • Solve novel problems without retraining
  • Adapt to new domains in real-time
  • Collaborate with perfect information sharing
  • Most critically: Improve themselves recursively

Every day without this infrastructure is a day your models can't build their own tools, can't improve their own capabilities, can't adapt to new challenges.

OpenAI's lead isn't just GPT-4's parameter count. It's the infrastructure that lets their models leverage tools, adapt behaviors, and solve complex multi-step problems. We're building that infrastructure as an independent layer that can make ANY model more capable – including Llama.

The ASI Race Is About Infrastructure, Not Just Models

The first organization to achieve ASI won't be the one with the biggest model – it'll be the one whose AI can improve itself without human intervention.

We've built the critical pieces:

  1. Tool Creation: AI that writes its own code (working today)
  2. Behavior Optimization: AI that improves its own prompts (working today)
  3. Architecture Design: AI that designs neural networks (working today)
  4. Recursive Improvement: AI that enhances its own capabilities (emerging now)

The gap between current AI and ASI isn't measured in parameters – it's measured in capabilities for self-improvement. We're systematically closing that gap.

About That $100M...

The platform we've built is the foundation for every superintelligence project you're funding.

While those researchers you're acquiring are designing the next generation of language models, we've built the platform that will let those models improve themselves – the critical capability for achieving ASI.

We accept payment in cryptocurrency, equity, or those really large checks Sam mentioned.

Sincerely,
The LIT Platform Team

We've built a comprehensive AI tooling ecosystem that enables dynamic tool creation, sophisticated AI behavior design, and real-time capability extension. For technical details, visit our repository or schedule a demo to see autonomous AI in action.

The AI Business Case Playbook: Securing Executive Buy-In

With AI investments delivering an average 3.7x ROI for generative AI implementations and top performers achieving 10.3x returns, the question isn't whether AI delivers value—it's how to quantify and communicate that value effectively. This playbook provides a comprehensive framework for building compelling ROI arguments for AI investment, covering the four pillars of AI value creation, ROI calculation methodologies, industry benchmarks, quick win examples, and strategies for effective executive presentations.

Industry Benchmarks

Industry-specific benchmarks provide valuable context for your AI business case, helping executives understand how your proposed investments compare to peer organizations. These benchmarks can strengthen your case by demonstrating that your projections align with real-world results.

Industry ROI Benchmarks

Manufacturing (275-400% ROI)

Manufacturing organizations typically see returns within 18 months, primarily through predictive maintenance, quality control automation, and supply chain optimization. The physical nature of manufacturing processes creates numerous high-value automation opportunities.

Financial Services (300-500% ROI)

Financial institutions achieve returns within 12-24 months through fraud detection, automated underwriting, personalized recommendations, and regulatory compliance automation. The data-rich environment of financial services creates fertile ground for AI applications.

Healthcare (250-400% ROI)

Healthcare organizations realize returns within 18-36 months via clinical decision support, administrative automation, and resource optimization. Longer timeframes reflect the regulated nature of healthcare and integration complexities with existing systems.

Retail (300-600% ROI)

Retail businesses see returns within 12-18 months through personalization engines, inventory optimization, and dynamic pricing. The direct connection to consumer behavior and purchasing decisions enables rapid value creation.

When using these benchmarks, select the most relevant industry category and timeframe for your specific use case. This helps set appropriate expectations while demonstrating that your projections are grounded in industry experience rather than speculation.

The Four Pillars of AI Value Creation

AI investments generate value across four key dimensions, each contributing a different proportion to the overall business impact. Understanding these pillars helps structure comprehensive business cases that capture AI's full potential.

Cost
Reduction
(40%)

The largest value driver typically comes from efficiency gains:

  • Process automation savings: 30-50% reduction in manual tasks
  • Error reduction: 15-75% decrease in quality defects
  • Resource optimization: 10-25% efficiency improvements
  • Maintenance cost reduction: 25-60% through predictive analytics

Revenue
Enhancement
(35%)

AI directly contributes to top-line growth through:

  • Personalized customer experiences: 10-30% sales increases
  • New product/service capabilities: 5-15% revenue growth
  • Market expansion opportunities: Variable based on industry
  • Pricing optimization: 5-15% margin improvements

Risk
Mitigation
(15%)

AI helps protect business value through:

  • Fraud prevention: 40-60% reduction in losses
  • Compliance automation: 30-50% cost reduction
  • Quality improvements: 20-40% defect prevention
  • Predictive risk management: Variable impact

Strategic
Advantage
(10%)

Long-term competitive benefits include:

  • Competitive differentiation through AI capabilities
  • Enhanced decision-making speed and accuracy
  • Innovation platform for future capabilities
  • Market positioning as technology leader

When building your AI business case, ensure you capture value across all four pillars to present a complete picture of potential returns. While cost reduction often provides the most immediate and measurable benefits, the strategic advantages can deliver exponential value over time.

ROI Calculation Framework

Quantifying AI's return on investment requires a structured approach that accounts for both immediate benefits and long-term value creation. The foundation of any AI business case is a clear, defensible ROI calculation.

Simple ROI = (Annual Benefits - Annual Costs) / Total Investment × 100%

While this basic formula provides a starting point, sophisticated AI business cases should incorporate a multi-year view that accounts for implementation timelines and benefit realization curves.

Year 1: 75% of projected benefits

Account for the implementation learning curve as systems are deployed and teams adapt to new workflows. First-year returns are typically conservative as the organization builds capability.

Year 2: 100% of projected benefits

Expect full operation and realization of initially projected benefits once systems are fully integrated and optimized for your specific business context.

Year 3: 125% of projected benefits

As organizations optimize and scale AI solutions, many discover additional use cases and efficiency gains beyond initial projections, creating compounding returns.

This conservative business case model helps manage executive expectations while still demonstrating compelling returns. It acknowledges the reality of implementation challenges while showing the progressive value creation typical of successful AI initiatives. When presenting to executives, emphasize that this phased approach represents a realistic path to value rather than overly optimistic projections.

Quick Win Examples: Customer Service Chatbot

Demonstrating concrete examples of AI implementations with clear ROI calculations helps executives visualize the potential value. Customer service chatbots represent one of the most accessible and high-return AI investments across industries.

$50K

First-Year Investment

Includes implementation costs, integration with existing systems, training, and ongoing maintenance for the first year of operation.
$130K

Annual Benefits

Derived from 40% reduction in customer service costs plus additional value from 24/7 availability improving customer satisfaction and retention.
160%

First-Year ROI

Even accounting for implementation time and learning curve, the chatbot delivers positive returns within the first year of deployment.
750%

Ongoing Annual ROI

After initial implementation, maintenance costs drop significantly while benefits continue to accrue, creating exceptional ongoing returns.

Customer service chatbots typically deliver value through multiple mechanisms:

Direct Cost Reduction

  • Reduced staffing requirements for routine inquiries
  • Lower cost per customer interaction (70-90% less than human agents)
  • Decreased training costs as chatbots handle standardized responses

Service Improvements

  • 24/7 availability without staffing constraints
  • Consistent quality of responses across all interactions
  • Immediate response times improving customer satisfaction
  • Multilingual support without additional resources

When presenting this example, emphasize that chatbots represent just one of many potential AI quick wins. The rapid implementation timeline and clear before/after metrics make them particularly effective for building organizational confidence in AI investments.

Quick Win Examples: Email Processing Automation

Email processing automation represents another high-ROI AI implementation that delivers rapid returns across various business functions. This use case demonstrates how AI can transform routine information processing tasks that consume significant employee time.

$40K

First-Year Investment

Covers implementation, integration with email systems, training the AI on company-specific email patterns, and ongoing maintenance.

$205K

Annual Benefits

Derived from 500 hours/month of time savings across the organization plus significant error reduction in email processing and routing.

412%

First-Year ROI

Even with implementation time, the solution delivers exceptional first-year returns by addressing a high-volume, labor-intensive process.

925%

Ongoing Annual ROI

After initial setup, maintenance costs decrease while the system continues to improve through learning, creating substantial ongoing returns.

Email processing automation creates value through multiple mechanisms:

Time Recapture

The average knowledge worker spends 28% of their workday managing email. Automation can reduce this by 40-60%, freeing skilled employees for higher-value activities. For an organization with 100 employees this represents over $840k in recaptured productive capacity annually.

Error Reduction

Manual email processing leads to misrouting, delayed responses, and missed action items. AI systems maintain consistent performance 24/7, reducing errors by 35-65% and improving compliance with service level agreements and response time commitments.

Scalability

Unlike manual processing, AI email systems can handle volume spikes without additional resources. This creates particular value for organizations with seasonal patterns or growth trajectories that would otherwise require hiring additional staff.

When presenting this example, emphasize that email processing represents a universal pain point across organizations, making it an excellent candidate for early AI implementation. The combination of clear before/after metrics and broad applicability across departments helps build cross-functional support for AI initiatives.

The Executive Presentation Framework

Successfully securing executive buy-in requires more than just solid numbers—it demands a strategic approach to communication that addresses both business priorities and potential concerns. This framework provides a proven structure for presenting AI investment proposals to senior leadership.

Start with the business problem, not the technology

Begin your presentation by clearly articulating the business challenge or opportunity, using language and metrics that resonate with executives. Frame AI as a solution to existing priorities rather than a technology in search of a problem. Connect your proposal directly to strategic objectives and KPIs that leadership already cares about.

Present conservative financial projections with sensitivity analysis

Provide realistic financial models that acknowledge implementation variables. Include sensitivity analysis showing outcomes under different scenarios (conservative, expected, optimistic). This demonstrates thorough analysis and builds credibility by acknowledging that results may vary based on implementation factors.

Include implementation timeline with clear milestones

Outline a phased implementation approach with specific milestones and success metrics for each stage. This demonstrates thoughtful planning and provides natural checkpoints for evaluating progress. Emphasize early wins that can build momentum and confidence in the broader initiative.

Address risks and mitigation strategies

Proactively identify potential implementation challenges and your plans to address them. This demonstrates foresight and builds confidence that you've considered potential obstacles. Include both technical risks and organizational change management considerations.

Compare to "do nothing" scenario costs

Quantify the cost of inaction, including missed opportunities, competitive disadvantages, and continuing inefficiencies. This creates urgency by framing AI investment not just as a new cost, but as an alternative to the hidden costs of maintaining the status quo.

Effective executive presentations balance detail with clarity, providing enough information to support decisions without overwhelming with technical specifics. Prepare a comprehensive appendix with additional details that can be referenced if questions arise, but keep the main presentation focused on business outcomes rather than technical implementation.

Building Your AI Business Case: Key Takeaways

Creating compelling AI investment proposals requires a strategic approach that balances technical possibilities with business realities. As you develop your AI business case, keep these essential principles in mind to maximize your chances of securing executive buy-in.

Focus on Business Outcomes

Frame AI investments in terms of specific business problems solved and quantifiable outcomes delivered. Connect directly to existing strategic priorities and KPIs that executives already care about. The technology should be secondary to the business value it creates.

Build Comprehensive Value Cases

Capture value across all four pillars: cost reduction (40%), revenue enhancement (35%), risk mitigation (15%), and strategic advantage (10%). While cost savings often provide the clearest initial ROI, the full value of AI emerges when all dimensions are considered.

Use Conservative Projections

Present realistic financial models that acknowledge implementation variables. The phased approach (75% benefits in Year 1, 100% in Year 2, 125% in Year 3) sets appropriate expectations while still demonstrating compelling returns.

Start Small, Scale Strategically

Begin with high-ROI quick wins like chatbots (160% first-year ROI) or email automation (412% first-year ROI) that demonstrate value rapidly. Use these successes to build organizational confidence and momentum for more ambitious AI initiatives. The examples provided show that even modest investments can deliver significant returns when properly targeted.

Address the Full Implementation Journey

Include not just the technology costs but also the organizational change management required for successful adoption. Outline clear implementation milestones, risk mitigation strategies, and a realistic timeline that acknowledges both technical and human factors in the deployment process.

Remember that securing executive buy-in is both an analytical and emotional process. While the numbers must be compelling, executives also need confidence in the implementation approach and alignment with strategic priorities. By following this playbook, you'll be well-positioned to build AI business cases that not only secure funding but set the foundation for successful implementation and value realization.

With AI investments delivering an average 3.7x ROI for generative AI implementations and top performers achieving 10.3x returns, organizations that develop this capability for building and communicating AI value will have a significant competitive advantage in the coming years.

Imperative vs. Declarative: A Concept That Built an Empire

For self-taught developers and those early in their careers, certain foundational concepts often remain unexamined. You pick up syntax, frameworks, and tools rapidly, but the underlying paradigms that shape how we think about code get overlooked.

These terms, imperative and declarative, sound academic but represent a simple distinction that transforms how you approach problems.

The Two Ways to Think About Problems

Imperative programming tells the computer exactly how to do something, step by step. Like giving someone furniture assembly instructions: "First, attach part A to part B using screw 3. Then insert dowel C into hole D."

Example:

const numbers = [1, 2, 3, 4, 5];

const doubled = [];
for (let i = 0; i < numbers.length; i++) {
  doubled.push(numbers[i] * 2);
}

Declarative programming describes what you want, letting the system figure out how. Like ordering at a restaurant: "I want a pepperoni pizza." You don't explain kneading dough or managing oven temperature.

Example:

const numbers = [1, 2, 3, 4, 5];

const doubled = numbers.map(n => n * 2);

Same result. But notice how the second approach abstracts away the loop management, index tracking, and array building. You declare your intent—"transform each number by doubling it"—and map handles the mechanics.

Why This Matters Beyond Syntax

The power becomes obvious when you scale up complexity. Consider drawing graphics:

Imperative (HTML Canvas): 

const ctx = canvas.getContext('2d');
ctx.beginPath();
ctx.rect(50, 50, 100, 75);
ctx.fillStyle = 'red';
ctx.fill();
ctx.closePath();

Declarative (SVG): 

<rect x="50" y="50" width="100" height="75" fill="red" />

The imperative version commands each drawing step. The declarative version simply states "there is a red rectangle here" and lets the renderer handle pixel manipulation, memory allocation, and screen updates.

The same pattern appears everywhere in modern development:

Database queries: SQL is declarative. You specify what data you want, not how to scan tables or optimize joins. 

SELECT name FROM users WHERE age > 25

Configuration management: Tools like Ansible let you declare desired system states rather than scripting installation steps. 

- name: Ensure Apache is installed and running
  service:
    name: apache2
    state: started

Modern JavaScript: Methods like filter, reduce, and find let you declare transformations instead of managing loops. 

// Instead of imperative loops
const adults = [];
for (let i = 0; i < users.length; i++) {
  if (users[i].age >= 18) {
    adults.push(users[i]);
  }
}

// Write declarative transformations
const adults = users.filter(user => user.age >= 18);

The Billion-Dollar Story

Now let me tell you how this simple principle reshaped the entire tech industry. In 1999, a buddy of mine, Google employee #21, tried to recruit me from Microsoft. "You probably don't even need to interview," he said, showing me their server racks filled with cheap consumer hardware. While competitors bought expensive fault-tolerant systems, Google was betting everything on commodity machines and a radical programming approach.

The imperative approach would be a coordination nightmare. Send chunk 1 to server 47, chunk 2 to server 134, wait for responses, handle server failures, retry failed chunks, merge partial results... Multiply that across thousands of machines and you get unmanageable complexity.

Instead, Google developed what became known as MapReduce; a declarative paradigm for distributed computing. Engineers could write: Here's my map function (extract words from web pages). Here's my reduce function (count word frequencies). Process the entire web. This framework handled all the imperative details: data distribution, failure recovery, load balancing, result aggregation. Engineers declared what they wanted computed. The system figured out how to coordinate thousands of servers.

This wasn't just elegant computer science. It was competitive advantage. While competitors struggled with complex distributed systems built on expensive hardware, Google's engineers focused on algorithms and data insights. Their declarative approach to distributed computing let them scale faster and cheaper than anyone thought possible.

What my friend was showing me in 1999, commodity hardware coordinated by smart software that abstracted away distributed complexity, was MapReduce in action, years before the famous 2004 paper. That paper didn't introduce a new concept; it documented the practices that had already powered Google's rise to dominance.

Beyond Obsolescence: The Modest Proposal for LLM-Native Workflow Automation

Our prior analysis, "The Beginning and End of LLM Workflow Software: How MCP Will Obsolesce Workflows," posited that Large Language Models (LLMs), amplified by the Model Context Protocol (MCP), will fundamentally reshape enterprise workflow automation. This follow-up expands on that foundational argument.

The impending shift is not one of outright elimination, but of a profound transformation. Rather than becoming entirely obsolete, the human-centric graphical user interface (GUI) for workflows will largely recede from direct human interaction, as the orchestration of processes evolves to be managed primarily by LLMs.

This critical pivot signifies a change in agency: the primary "user" of workflow capabilities shifts from human to AI. Here, we lay out a modest proposal for a reference architecture that brings this refined vision to life, detailing how LLMs will interact with and harness these next-generation workflow systems.

The Modest Proposal: An LLM-Native Workflow Architecture

Our vision for the future of workflow automation centers on LLMs as the primary orchestrators of processes, with human interaction occurring at a much higher, conversational level. This shifts the complexity away from the human and into the intelligent automation system itself.

MCP Servers: The Secure Hands of the LLM

The foundation of this architecture is the Model Context Protocol (MCP), or similar secure resource access protocols. At Lit.ai, our approach is built on a fundamental philosophy that ensures governance and audibility: any action a user initiates via our platform ultimately executes as that user on the host system. For instance, when a user uploads a file through our web interface, a ls -l command reveals that file is literally "owned" by that user on disk. Similarly, when they launch a training process, a data build, or any other compute-intensive task, a ps aux command reveals that the process was launched by that user's identity, not a shared service account. This granular control is seamlessly integrated with enterprise identity and access management through Keycloak, enabling features like single sign-on (SSO) and federated security. You can delve deeper into our "Execute as User" principle here: https://docs.lit.ai/reference/philosophy/#execute-as-user-a-foundation-of-trust-and-control.

We've now seamlessly extended this very philosophy to our MCP servers. When launched for LLM interactions, these servers inherit the user's existing permissions and security context, ensuring the LLM's actions are strictly governed by the user's defined access rights. This isn't a speculative new security model for AI; it's an intelligent evolution of established enterprise security practices. All LLM-initiated actions are inherently auditable through existing system logs, guaranteeing accountability and adherence to the principle of least privilege.

The LLM's Workflow Interface: Submerged and Powerful

In this new era, legacy visual workflow software won't vanish entirely; instead, it transforms into sophisticated tools primarily used by the LLM. Consider an LLM's proven ability to generate clean JSON documents from natural language prompts. This is precisely how it will interact with the underlying workflow system.

This LLM-native interface offers distinct advantages over traditional human GUIs, because it's designed for programmatic interaction, not visual clicks and drags:

  • Unconstrained by Human UIs: The LLM doesn't need to visually parse a flowchart or navigate menus. It interacts directly with the workflow system's deepest configuration layers. This means the workflow tool's capabilities are no longer limited by what a human developer could represent in a GUI. For example, instead of waiting for a vendor to build UI components for a new property or function, the LLM can define and leverage these dynamically. The underlying workflow definition could be a flexible data structure like a JSON document, infinitely extensible on the fly by the LLM.

  • Unrivaled Efficiency: An LLM can interpret and generate the precise underlying code, API calls, or domain-specific language that defines the process. This direct programmatic access is orders of magnitude more efficient than any human-driven clicks and drags. Imagine the difference between writing machine code directly versus meticulously configuring a complex circuit board by hand—the LLM operates at a vastly accelerated conceptual level.

  • Dynamic Adaptation and Reactive Feature Generation: The LLM won't just create workflows from scratch; it will dynamically modify them in real-time. This includes its remarkable ability to write and integrate code changes on the fly to add features to a live workflow, or adapt to unforeseen circumstances. This provides a reactive, agile automation layer that can self-correct and enhance processes as conditions change, all without human intervention in a visual design tool.

  • Autonomous Optimization: Leveraging its analytical capabilities, the LLM could continuously monitor runtime data, identify bottlenecks or inefficiencies within the workflow's execution, and even implement optimizations to the process's internal logic. This moves from human-initiated process improvement to continuous, AI-driven refinement.

This approach creates a powerful separation: humans define what needs to happen through natural language, and the LLM handles how it happens, managing the intricate details of process execution within its own highly efficient, automated interface.

Illustrative Scenarios: Realizing Value with LLM-Native Workflows

Let's look at how this translates into tangible value creation:

Empowering Customer Service with Conversational Data Access

Imagine a customer service representative (CSR) on a call. In a traditional setup, the CSR might navigate a legacy Windows application, click through multiple tabs, copy-paste account numbers, and wait for various system queries to retrieve customer data. This is often clunky, slow, and distracting.

In an LLM-native environment, the CSR simply asks their AI assistant: "What is John Doe's current account balance and recent purchase history for product X?" Behind the scenes, the LLM, via MCP acting as the CSR, seamlessly accesses the CRM, payment system, and order database. It orchestrates the necessary API calls, pulls disparate data, and synthesizes a concise, relevant answer instantly. The entire "workflow" of retrieving, joining, and presenting this data happens invisibly, managed by the LLM, eliminating manual navigation and dramatically improving customer experience.

Accelerating Marketing Campaigns with AI Orchestration

Consider a marketing professional launching a complex, multi-channel campaign. Historically, this might involve using a dedicated marketing automation platform to visually design a workflow: dragging components for email sends, social media posts, ad placements, and follow-up sequences. Each component needs manual configuration, integration setup, and testing.

With an LLM-native approach, the marketing person converses with the AI: "Launch a campaign for our new Q3 product, target customers in segments A and B, include a personalized email sequence, a social media push on LinkedIn and X, and a retargeting ad on Google Ads. If a customer clicks the email link, send a follow-up SMS."

The LLM interprets this narrative. Using its access to marketing platforms via MCP, it dynamically constructs the underlying "workflow"—configuring the email platform, scheduling social posts, setting up ad campaigns, and integrating trigger-based SMS. If the marketing team later says, "Actually, let's add TikTok to that social push," the LLM seamlessly updates the live campaign's internal logic, reacting and adapting in real-time, requiring no manual GUI manipulation.

Dynamic Feature Enhancement for Core Business Logic

Imagine a core business process, like loan application review. Initially, the LLM-managed workflow handles standard credit checks and document verification. A new regulation requires a specific new bankruptcy check and a conditional review meeting for certain applicants.

Instead of a developer manually coding changes into a workflow engine, a subject matter expert (SME) simply tells the LLM: "For loan applications, also check if the applicant has had a bankruptcy in the last five years. If so, automatically flag the application and schedule a review call with our financial advisor team, ensuring it respects their calendar availability."

The LLM, understanding the existing process and having access to the bankruptcy database API and scheduling tools via MCP, dynamically writes or modifies the necessary internal code for the loan review "workflow." It adds the new conditional logic and scheduling steps, demonstrating its reactive ability to enhance core features without human intervention in a visual design tool.

Human Expertise: The Indispensable LLM Coaches

In this evolved landscape, human expertise isn't diminished; it's transformed and elevated. The "citizen developer" who mastered a specific GUI gives way to the LLM Coach or Context Engineer. These are the subject matter experts (SMEs) within an organization who deeply understand their domain, the organization's data, and its unique business rules. Their role becomes one of high-level guidance:

  • Defining Context: Providing the LLM with the nuanced information it needs about available APIs, data schemas, and precise business rules.

  • Prompt Strategy & Oversight: Guiding the LLM in structuring effective prompts and conversational patterns, and defining the overarching strategy for how the LLM interacts with its context to achieve optimal results. This involves ensuring the LLM understands and applies the best practices for prompt construction, even as it increasingly manages the literal generation of those prompts itself.

  • Feedback and Coaching: Collaborating with the LLM to refine its behavior, validate its generated logic, and ensure it accurately meets complex requirements.

  • Strategic Oversight: Auditing LLM-generated logic and ensuring compliance, especially for critical functions.

This evolution redefines human-AI collaboration, leveraging the strengths of both. It ensures that the profound knowledge held by human experts is amplified, not replaced, by AI.

Anticipating Counterarguments and Refutations

We're aware that such a fundamental shift invites scrutiny. Let's address some common counterarguments head-on:

"This is too complex to set up initially."

While the initial phase requires defining the LLM's operational context – exposing APIs, documenting data models, and ingesting business rules – this is a one-time strategic investment in foundational enterprise knowledge. This effort shifts from continuous, tool-specific GUI configuration (which itself is complex and time-consuming) to building a reusable, LLM-consumable knowledge base. Furthermore, dedicated "LLM Coaches" (SMEs) will specialize in streamlining this process, making the setup efficient and highly valuable.

"What about the 'black box' problem for critical processes?"

For critical functions where deterministic behavior and explainability are paramount, our architecture directly addresses this. The LLM is empowered to generate determinate, auditable code (e.g., precise Python functions or specific machine learning models) for these decision points. This generated code can be inspected, verified, and integrated into existing compliance frameworks, ensuring transparency where it matters most. The "black box" is no longer the LLM's inference, but the transparent, verifiable code it outputs.

"Humans need visual workflows to understand processes."

While humans do value visualizations, these will become "on-demand" capabilities, generated precisely when needed. The LLM can produce contextually relevant diagrams (like Mermaid diagrams), data visualizations, or flowcharts based on natural language queries. The visual representation becomes a result of the LLM's understanding and orchestration, not the primary, cumbersome means of defining it. Users won't be forced to manually configure diagrams; they'll simply ask the LLM to show them the process.

The Dawn of LLM-Native Operations

The future of workflow automation isn't about better diagrams and drag-and-drop interfaces for humans. It's about a fundamental transformation where intelligent systems, driven by natural language, directly orchestrate the intricate processes of the enterprise. Workflow tools, rather than being obsolesced, will evolve to serve a new primary user: the LLM itself.

The Beginning and End of LLM Workflow Software: How MCP Will Obsolesce Workflows

In the rapidly evolving landscape of enterprise software, we're witnessing the meteoric rise of workflow automation tools. These platforms promise to streamline operations through visual interfaces where users can design, implement, and monitor complex business processes. Yet despite their current popularity, these GUI-based workflow solutions may represent the last generation of their kind—soon to be replaced by more versatile Large Language Model (LLM) interfaces.

The Current Workflow Software Boom

The workflow automation market is experiencing unprecedented growth, projected to reach 78.8 billion USD by 2030 with a staggering 23.1% compound annual growth rate. This explosive expansion is evident in both funding activity and market adoption: Workato secured a 200 million USD Series E round at a $5.7 billion valuation, while established players like ServiceNow and Appian continue to report record subscription revenues.

A quick glance at a typical workflow builder interface reveals the complexity these tools embrace:

alt text

The landscape is crowded with vendors aggressively competing for market share:

  • Enterprise platforms: ServiceNow, Pega, Appian, and IBM Process Automation dominate the high-end market, offering comprehensive solutions tightly integrated with their broader software ecosystems.
  • Integration specialists: Workato, Tray.io, and Zapier focus specifically on connecting disparate applications through visual workflow builders, catering to the growing API economy.
  • Emerging players: Newer entrants like Bardeen, n8n, and Make (formerly Integromat) are gaining traction with innovative approaches and specialized capabilities.

This workflow automation boom follows a familiar pattern we've seen before. Between 2018 and 2022, Robotic Process Automation (RPA) experienced a similar explosive growth cycle. Companies like UiPath reached a peak valuation of $35 billion before a significant market correction as limitations became apparent. RPA promised to automate routine tasks by mimicking human interactions with existing interfaces—essentially screen scraping and macro recording at an enterprise scale—but struggled with brittle connections, high maintenance overhead, and limited adaptability to changing interfaces.

Today's workflow tools attempt to address these limitations by focusing on API connections rather than UI interactions, but they still follow the same fundamental paradigm: visual programming interfaces that require specialized knowledge to build and maintain.

So why are organizations pouring billions into these platforms despite the lessons from RPA? Several factors drive this investment:

  • Digital transformation imperatives: COVID-19 dramatically accelerated organizations' need to automate processes as remote work became essential and manual, paper-based workflows proved impossible to maintain.
  • The automation gap: Companies recognize the potential of AI and automation but have lacked accessible tools to implement them across the organization without heavy IT involvement.
  • Democratization promise: Workflow tools market themselves as empowering "citizen developers"—business users who can automate their own processes without coding knowledge.
  • Pre-LLM capabilities: Until recently, organizations had few alternatives for process automation that didn't require extensive software development.

What we're witnessing is essentially a technological stepping stone—organizations hungry for AI-powered results before true AI was ready to deliver them at scale. But as we'll see, that technological gap is rapidly closing, with profound implications for the workflow software category.

Why LLMs Will Disrupt Workflow Software

While current workflow tools represent incremental improvements on decades-old visual programming paradigms, LLMs offer a fundamentally different approach—one that aligns with how humans naturally express process logic and intent. The technical capabilities enabling this shift are advancing rapidly, creating the conditions for widespread disruption.

The Technical Foundation: Resource Access Protocols

The key technical enabler for LLM-driven workflows is the development of secure protocols that allow these models to access and manipulate resources. Model Context Protocol (MCP) represents one of the most promising approaches:

MCP provides a standardized way for LLMs to:

  • Access data from various systems through controlled APIs
  • Execute actions with proper authentication and authorization
  • Maintain context across multiple interactions
  • Document actions taken for compliance and debugging

Unlike earlier attempts at AI automation, MCP and similar protocols solve the "last mile" problem by creating secure bridges between conversational AI and the systems that need to be accessed or manipulated. Major cloud providers are already implementing variations of these protocols, with Microsoft's Azure AI Actions, Google's Gemini API, and Anthropic's Claude Tools representing early implementations.

The proliferation of these standards means that instead of building custom integrations for each workflow tool, organizations can create a single set of LLM-compatible APIs that work across any AI interface.

Natural Language vs. GUI Interfaces

The cognitive load difference between traditional workflow tools and LLM interfaces becomes apparent when comparing approaches to the same problem:

Traditional Workflow Tool Process
  1. Open workflow designer application
  2. Create a new workflow and name it
  3. Drag "Trigger" component (Customer Signup)
  4. Configure webhook or database monitor
  5. Drag "HTTP Request" component
  6. Configure endpoint URL for credit API
  7. Add authentication parameters (API key, tokens)
  8. Add request body parameters and format
  9. Connect to "JSON Parser" component
  10. Define schema for response parsing
  11. Create variable for credit score
  12. Add "Decision" component
  13. Configure condition (score < 600)
  14. For "True" path, add "Notification" component
  15. Configure recipients, subject, and message template
  16. Add error handling for API timeout
  17. Add error handling for data format issues
  18. Test with sample data
  19. Debug connection issues
  20. Deploy to production environment
  21. Configure monitoring alerts
LLM Approach
When a new customer signs up, retrieve their credit score from our API, 
store it in our database, and if the score is below 600, notify the risk 
assessment team.

The workflow tool approach requires not only understanding the business logic but also learning the specific implementation patterns of the tool itself. Users must know which components to use, how to properly connect them, and how to configure each element—skills that rarely transfer between different workflow platforms.

Dynamic Adaptation Through Conversation

Real business processes rarely remain static. Consider how process changes propagate in each paradigm:

Traditional Workflow Change Process
  1. Open existing workflow in designer
  2. Identify components that need modification
  3. Add new components for bankruptcy check
  4. Configure API connection to bankruptcy database
  5. Add new decision branch
  6. Connect positive result to new components
  7. Add calendar integration component
  8. Configure meeting details and attendees
  9. Update documentation to reflect changes
  10. Redeploy updated workflow
  11. Test all paths, including existing functionality
  12. Update monitoring for new failure points
LLM Approach
Actually, let's also check if they've had a bankruptcy in the last five 
years, and if so, automatically schedule a review call with our financial 
advisor team.

The LLM simply incorporates the new requirement conversationally. Behind the scenes, it maintains a complete understanding of the existing process and extends it appropriately—adding the necessary API calls, conditional logic, and scheduling actions without requiring the user to manipulate visual components.

Early implementations of this approach are already appearing. GitHub Copilot for Docs can update software configuration by conversing with developers about their intentions, rather than requiring them to parse documentation and make manual changes. Similarly, companies like Adept are building AI assistants that can operate existing software interfaces based on natural language instructions.

Self-Healing Systems: The Maintenance Advantage

Perhaps the most profound advantage of LLM-driven workflows is their ability to adapt to changing environments without breaking. Traditional workflows are notoriously brittle:

Traditional Workflow Failure Scenarios:

  • An API endpoint changes its structure
  • A data source modifies its authentication requirements
  • A third-party service deprecates a feature
  • A database schema is updated
  • Operating system or runtime dependencies change

When these changes occur, traditional workflows break and require manual intervention. Someone must diagnose the issue, understand the change, modify the workflow components, test the fixes, and redeploy. This maintenance overhead is substantial—studies suggest organizations spend 60-80% of their workflow automation resources on maintenance rather than creating new value.

LLM-Driven Workflow Adaptation: LLMs with proper resource access can automatically adapt to many changes:

  • When an API returns errors, the LLM can examine documentation, test alternative approaches, and adjust parameters
  • If authentication requirements change, the LLM can interpret error messages and modify its approach
  • When services deprecate features, the LLM can find and implement alternatives based on its understanding of the underlying intent
  • Changes in database schemas can be discovered and accommodated dynamically
  • Environmental changes can be detected and worked around

Rather than breaking, LLM-driven workflows degrade gracefully and can often self-heal without human intervention. When they do require assistance, the interaction is conversational:

User: The customer onboarding workflow seems to be failing at the credit check 
step.
LLM: I've investigated the issue. The credit API has changed its response 
format. I've updated the workflow to handle the new format. Would you like 
me to show you the specific changes I made?

This self-healing capacity drastically reduces maintenance overhead and increases system reliability. Organizations using early LLM-driven processes report up to 70% reductions in workflow maintenance time and significantly improved uptime.

Compliance and Audit Superiority

Perhaps counterintuitively, LLM-driven workflows can provide superior compliance capabilities. Several financial institutions are already piloting LLM systems that maintain comprehensive audit logs that surpass traditional workflow tools:

  • Granular Action Logging: Every step, decision point, and data access is logged with complete context
  • Natural Language Explanations: Each action includes an explanation of why it was taken
  • Cryptographic Verification: Logs can be cryptographically signed and verified for tamper detection
  • Full Data Lineage: Complete tracking of where data originated and how it was transformed
  • Semantic Search: Compliance teams can query logs using natural language questions

A major U.S. bank recently compared their existing workflow tool's audit capabilities with a prototype LLM-driven system and found the LLM approach provided 3.5x more detailed audit information with 65% less storage requirements, due to the elimination of redundant metadata and more efficient logging.

Visualization On Demand

For scenarios where visual representation is beneficial, LLMs offer a significant advantage: contextually appropriate visualizations generated precisely when needed.

Rather than being limited to pre-designed dashboards and reports, users can request visualizations tailored to their current needs:

User: Show me a diagram of how the customer onboarding process changes with 
the new bankruptcy check.

LLM: Generates a Mermaid diagram showing the modified process flow with the 
new condition highlighted

User: How will this affect our approval rates based on historical data?

LLM: Generates a bar chart showing projected approval rate changes based on 
historical bankruptcy data

Companies like Observable and Vercel are already building tools that integrate LLM-generated visualizations into business workflows, allowing users to create complex data visualizations through conversation rather than manual configuration.

Current State of Adoption

While the technical capabilities exist, we're still in the early stages of this transition. Rather than presenting hypothetical examples as established successes, it's more accurate to examine how organizations are currently experimenting with LLM-driven workflow approaches:

  • Prototype implementations: Several companies are building prototype systems that use LLMs to orchestrate workflows, but these remain largely experimental and haven't yet replaced enterprise-wide workflow systems.
  • Augmentation rather than replacement: Most organizations are currently using LLMs to augment existing workflow tools—helping users configure complex components or troubleshoot issues—rather than replacing the tools entirely.
  • Domain-specific applications: The most successful early implementations focus on narrow domains with well-defined processes, such as content approval workflows or customer support triage, rather than attempting to replace entire workflow platforms.
  • Hybrid approaches: Organizations are finding success with approaches that combine traditional workflow engines with LLM interfaces, allowing users to interact conversationally while maintaining the robustness of established systems.

While we don't yet have large-scale case studies with verified metrics showing complete workflow tool replacement, the technological trajectory is clear. As LLM capabilities continue to improve and resource access protocols mature, the barriers to adoption will steadily decrease.

Investment Implications

The disruption of workflow automation by LLMs isn't a gradual shift—it's happening now. For decision-makers, this isn't about careful transitions or hedged investments; it's about immediate and decisive action to avoid wasting resources on soon-to-be-obsolete technology.

Halt Investment in Traditional Workflow Tools Immediately

Stop signing or renewing licenses for traditional workflow automation platforms. These systems will be obsolete within weeks, not years. Any new investment in these platforms represents resources that could be better allocated to LLM+MCP approaches. If you've recently purchased licenses, investigate termination options or ways to repurpose these investments.

Redirect Resources to LLM Infrastructure

Immediately reallocate budgets from workflow software to: - Enterprise-grade LLM deployment on your infrastructure - Implementation of MCP or equivalent protocols - API development for all internal systems - Prompt engineering training for existing workflow specialists

Install LLM+MCP on Every Desktop Now

Rather than planning gradual rollouts, deploy LLM+MCP capabilities across your organization immediately. Every day that employees continue to build workflows in traditional tools is a day of wasted effort creating systems that will need to be replaced. Local or server-based LLMs with proper resource access should become standard tools alongside word processors and spreadsheets.

Retrain Teams for the New Paradigm

Your workflow specialists need to become prompt engineers—not next quarter, but this week: - Cancel scheduled workflow tool training - Replace with intensive prompt engineering workshops - Focus on teaching conversational process design rather than visual programming - Develop internal guides for effective LLM workflow creation

For organizations with existing contracts for workflow platforms: - Review termination clauses and calculate the cost of early exits - Investigate whether remaining license terms can be applied to API access rather than visual workflow tools - Consider whether vendors might offer transitions to their own LLM offerings in lieu of contracted services

Vendors: Pivot or Perish

For workflow automation companies, there's no time for careful transitions: - Immediately halt development on visual workflow designers - Redirect all engineering resources to LLM interfaces and connectors - Open all APIs and create comprehensive documentation for LLM interaction - Develop prompt libraries that encapsulate existing workflow patterns

The AI-assisted development cycle is accelerating innovation at unprecedented rates. What would have taken years is now happening in weeks. Organizations that try to manage this as a gradual transition will find themselves outpaced by competitors who embrace the immediate shift to LLM-driven processes.

Our Own Evolution

We need to acknowledge our own journey in this space. At Lit.ai, we initially invested in building the Workflow Canvas - a visual tool for designing LLM-powered workflows that made the technology more accessible. We created this product with the belief that visual workflow builders would remain essential for orchestrating complex LLM interactions.

However, our direct experience with customers and the rapid evolution of LLM capabilities has caused us to reassess this position. The very technology we're building is becoming sophisticated enough to make our own workflow canvas increasingly unnecessary for many use cases. Rather than clinging to this approach, we're now investing heavily in Model Context Protocol (MCP) and direct LLM resource access.

This pivot represents our commitment to following the technology where it leads, even when that means disrupting our own offerings. We believe the most valuable contribution we can make isn't building better visual workflow tools, but rather developing the connective tissue that allows LLMs to directly access and manipulate the resources they need to execute workflows without intermediary interfaces.

Our journey mirrors what we expect to see across the industry - an initial investment in workflow tools as a stepping stone, followed by a recognition that the real value lies in direct LLM orchestration with proper resource access protocols.

Timeline and Adoption Considerations

While the technical capabilities enabling this shift are rapidly advancing, several factors will influence adoption timelines:

Enterprise Inertia

Large organizations with established workflow infrastructure and trained teams will transition more slowly. Expect these environments to adopt hybrid approaches initially, where LLMs complement rather than replace existing workflow tools.

High-Stakes Domains

Industries with mission-critical workflows (healthcare, finance, aerospace) will maintain traditional interfaces longer, particularly for processes with significant safety or regulatory implications. However, even in these domains, LLMs will gradually demonstrate their reliability for increasingly complex tasks.

Security and Control Concerns

Organizations will need to develop comfort with LLM-executed workflows, particularly regarding security, predictability, and control. Establishing appropriate guardrails and monitoring will be essential for building this confidence.

Conclusion

The current boom in workflow automation software represents the peak of a paradigm that's about to be disrupted. As LLMs gain direct access to resources and demonstrate their ability to understand and execute complex processes through natural language, the value of specialized GUI-based workflow tools will diminish.

Forward-thinking organizations should prepare for this shift by investing in API infrastructure, LLM integration capabilities, and domain-specific knowledge engineering rather than committing deeply to soon-to-be-legacy workflow platforms. The future of workflow automation isn't in better diagrams and drag-drop interfaces—it's in the natural language interaction between users and increasingly capable AI systems.

In fact, this very article demonstrates the principle in action. Rather than using a traditional publishing workflow tool with multiple steps and interfaces, it was originally drafted in Google Docs, then an LLM was instructed to: 

Translate this to markdown, save it to a file on the local disk, execute a 
build, then upload it to AWS S3.
The entire publishing workflow—format conversion, file system operations, build process execution, and cloud deployment—was accomplished through a simple natural language request to an LLM with the appropriate resource access, eliminating the need for specialized workflow interfaces.

This perspective challenges conventional wisdom about enterprise software evolution. Decision-makers who recognize this shift early will gain significant advantages in operational efficiency, technology investment, and organizational agility.

The Rising Value of Taxonomies in the Age of LLMs

Introduction

Large Language Models (LLMs) are growing the demand for structured data, creating a significant opportunity for companies specializing in organizing that data. This article explores how this trend is making expertise in taxonomies and data-matching increasingly valuable for businesses seeking to utilize LLMs effectively.

LLMs Need Structure

LLMs excel at understanding and generating human language. However, they perform even better when that language is organized in a structured way, which improves accuracy, consistency, and reliability. Consider this: Imagine asking an LLM to find all research papers related to a specific protein interaction in a particular type of cancer. If the LLM only has access to general scientific abstracts and articles, it might provide a broad overview of cancer research but struggle to pinpoint the highly specific information you need. You might get a lot of information about cancer in general, but not a precise list of papers that focus on the specific protein interaction.

However, if the LLM has access to a structured database of scientific literature with detailed metadata and relationships, it can perform much more targeted research. This database would include details like:

  • Protein names and identifiers
  • Cancer types and subtypes
  • Experimental methods and results
  • Genetic and molecular pathways
  • Relationships to other research papers and datasets

With this structured data, the LLM can quickly identify the relevant papers, analyze their findings, and provide a more focused and accurate summary of the research. This structured approach ensures that the LLM considers critical scientific details and avoids generalizations that might not be relevant to the specific research question. Taxonomies and ontologies are essential for organizing and accessing this kind of complex scientific information.

Large Language Models often benefit significantly from a technique called Retrieval-Augmented Generation (RAG). RAG involves retrieving relevant information from an external knowledge base and providing it to the LLM as context for generating a response. However, RAG systems are only as effective as the data they retrieve. Without well-structured data, the retrieval process can return irrelevant, ambiguous, or incomplete information, leading to poor LLM output. This is where taxonomies, ontologies, and metadata become crucial. They provide the 'well-defined scope' and 'high-quality retrievals' that are essential for successful RAG implementation. By organizing information into clear categories, defining relationships between concepts, and adding rich context, taxonomies enable RAG systems to pinpoint the most relevant data and provide LLMs with the necessary grounding for accurate and insightful responses.

To address these challenges and provide the necessary structure, we can turn to taxonomies. Let's delve into what exactly a taxonomy is and how it can benefit LLMs.

What is a Taxonomy

A taxonomy is a way of organizing information into categories and subcategories. Think of it as a hierarchical classification system. A good example is the biological taxonomy used to classify animals. For instance, red foxes are classified as follows:

  • Domain: Eukarya (cells with nuclei)
  • Kingdom: Animalia (all animals)
  • Phylum: Chordata (animals with a backbone)
  • Class: Mammalia (mammals)
  • Order: Carnivora (carnivores)
  • Family: Canidae (dogs)
  • Genus: Vulpes (foxes)
  • Species: Vulpes Vulpes (red fox)

alt text Annina Breen, CC BY-SA 4.0, via Wikimedia Commons

This hierarchical structure shows how we move from a very broad category (all animals) to a very specific one (Red Fox). Just like this animal taxonomy, other taxonomies organize information in a structured way.

Taxonomies provide structure by:

  • Improving Performance: Taxonomies help LLMs focus on specific areas, reducing the risk of generating incorrect or nonsensical information and improving the relevance of their output.
  • Facilitating Data Integration: Taxonomies can integrate data from various sources, providing LLMs with a more comprehensive and unified view of information. This is crucial for tasks that require broad knowledge and context.
  • Providing Contextual Understanding: Taxonomies offer a framework for understanding the relationships between concepts, enabling LLMs to generate more coherent and contextually appropriate responses.

Types of Taxonomies

There are several different types of taxonomies, each with its own strengths and weaknesses, and each relevant to how LLMs can work with data:

Hierarchical Taxonomies: Organize information in a tree-like structure, with broader categories at the top and more specific categories at the bottom. This is the most common type, often used in library classification or organizational charts. For LLMs, this provides a clear, nested structure that aids in understanding relationships and navigating data.

Faceted Taxonomies: Allow information to be categorized in multiple ways, enabling users to filter and refine their searches. Think of e-commerce product catalogs with filters for size, color, and price. This is particularly useful for LLMs that need to handle complex queries and provide highly specific results, as they can leverage multiple facets to refine their output.

Polyhierarchical Taxonomies: A type of hierarchical taxonomy where a concept can belong to multiple parent categories. For example, "tomato" could be classified under both "fruits" and "red foods." This allows LLMs to understand overlapping categories and handle ambiguity in classification.

Associative Taxonomies: Focus on relationships between concepts, rather than just hierarchical structures. For example, a taxonomy of "car" could include terms like "wheel," "engine," "road," and "transportation," highlighting the interconnectedness of these concepts. This helps LLMs understand the broader context and semantic relationships between terms, improving their ability to generate coherent and relevant responses.

Ultimately, the increasing reliance on LLM-generated content necessitates the implementation of well-defined taxonomies to unlock its full potential. The specific type of taxonomy may vary depending on the application, but the underlying principle remains: taxonomies are essential for enhancing the value and utility of LLM outputs.

LLMs and Internal Knowledge Representation

While we've discussed various types of external taxonomies, it's important to note that LLMs also develop their own internal representations of knowledge. These internal representations differ significantly from human-curated taxonomies and play a crucial role in how LLMs process information.

One way LLMs represent knowledge is through word vectors. These are numerical representations of words where words with similar meanings are located close to each other in a multi-dimensional space. For example, the relationship "king - man + woman = queen" can be captured through vector arithmetic, demonstrating how LLMs can represent semantic relationships.

alt text Ben Vierck, Word Vector Illustration, CC0 1.0

The word vector graph illustrates semantic relationships captured by LLMs using numerical representations of words. Each word is represented as a vector in a multi-dimensional space. In this example, the vectors for 'royal,' 'king,' and 'queen' originate at the coordinate (0,0), depicting their positions in this space. The vector labeled 'man' extends from the end of the 'royal' vector to the end of the 'king' vector, while the vector labeled 'woman' extends from the end of the 'royal' vector to the end of the 'queen' vector. This arrangement demonstrates how LLMs can represent semantic relationships such as 'king' being 'royal' plus 'man,' and 'queen' being 'royal' plus 'woman.' The spatial relationships between these vectors reflect the conceptual relationships between the words they represent.

However, these internal representations, unlike human-curated taxonomies, are:

  • Learned, Not Curated: Acquired through exposure to massive amounts of text data, rather than through a process of human design and refinement. This means the LLM infers relationships, rather than having them explicitly defined.
  • Unstructured: The relationships learned by LLMs may not always fit into a clear, hierarchical structure.
  • Context-Dependent: The meaning of a word or concept can vary depending on the surrounding text, making it difficult for LLMs to consistently apply a single, fixed categorization.
  • Incomplete: It's important to understand that LLMs don't know what they don't know. They might simply be missing knowledge of specific domains or specialized terminology that wasn't included in their training data.

This is where taxonomies become crucial. They provide an external, structured framework that can:

  • Constrain LLM Output: By mapping LLM output to a defined taxonomy, we can ensure that the information generated is consistent, accurate, and relevant to a specific domain.
  • Ground LLM Knowledge: Taxonomies can provide LLMs with access to authoritative, curated knowledge that may be missing from their training data.
  • Bridge the Gap: Taxonomies can bridge the gap between the unconstrained, often ambiguous language that humans use and the more structured, formal representations that LLMs can effectively process.

Taxonomies as Service Providers

Companies that specialize in creating and managing taxonomies and developing metadata schemas and ontologies to complement taxonomies are well-positioned to become key service providers in the LLM ecosystem. Their existing expertise in organizing information and structuring data makes them uniquely qualified to help businesses harness LLMs effectively.

For example, companies that specialize in organizing complex data for specific industries, such as healthcare or finance, often create proprietary systems to analyze and categorize information for their clients. In the healthcare sector, a company might create a proprietary methodology for evaluating healthcare plan value, categorizing patients based on risk factors and predicting healthcare outcomes. In the realm of workforce development, a company might develop a detailed taxonomy of job skills, enabling employers to evaluate their current workforce capabilities and identify skill gaps. This same taxonomy can also empower job seekers to understand the skills needed for emerging roles and navigate the path to acquiring them. These companies develop expertise in data acquisition, market understanding, and efficient data processing to deliver valuable insights.

Companies that specialize in creating and managing taxonomies are not only valuable for general LLM use but also for improving the effectiveness of Retrieval-Augmented Generation systems. RAG's limitations, such as retrieving irrelevant or ambiguous information, often stem from underlying data organization issues. Taxonomy providers can address these issues by creating robust knowledge bases, defining clear data structures, and adding rich metadata. This ensures that RAG systems can retrieve the most relevant and accurate information, thereby significantly enhancing the quality of LLM outputs. In essence, taxonomy experts can help businesses transform their RAG systems from potentially unreliable tools into highly effective knowledge engines.

Strategic Opportunities for Taxonomy Providers in the LLM Era

The rapid advancement and adoption of LLMs are driving an increase in demand for automated content generation. Businesses are increasingly looking to replace human roles with intelligent agents capable of handling various tasks, from customer service and marketing to data analysis and research. This drive towards agent-driven automation creates a fundamental need for well-structured data and robust taxonomies. Companies specializing in these areas are strategically positioned to capitalize on this demand.

Here's how taxonomy companies can leverage this market shift:

1. Capitalizing on the Content Generation Boom:

Demand-Driven Growth: The primary driver will be the sheer volume of content that businesses want to generate using LLMs and agents. Taxonomies are essential to ensure this content is organized, accurate, and aligned with specific business needs. Emphasize that the core opportunity lies in meeting this growing demand.

Agent-Centric Focus: Highlight that the demand is not just for general content but for content that powers intelligent agents. This requires taxonomies that are not just broad but highly specific and contextually rich.

2. Building Partnerships:

The surge in demand for LLM-powered applications and intelligent agents is creating a wave of new organizations focused on developing these solutions. Many of these companies will need specialized data, including job skills taxonomies, to power their agents effectively. This presents a unique opportunity for the job skills taxonomy provider to forge strategic partnerships.

Addressing the "Build vs. Buy" Decision: Many new agent builders will face the decision of whether to build their own skills taxonomy from scratch or partner with an existing provider. Given the rapid pace of LLM development and the complexity of creating and maintaining a robust taxonomy, partnering often proves to be the most efficient and cost-effective route. The taxonomy company can highlight the advantages of partnering:

  • Faster time to market
  • Higher quality data
  • Ongoing updates and maintenance

By targeting these emerging agent-building organizations, the job skills taxonomy company can capitalize on the growing demand for LLM-powered solutions and establish itself as a critical data provider in the evolving AI-driven workforce development landscape. This approach focuses on the new opportunities created by the LLM boom, rather than the existing operations of the taxonomy provider.

Seamless Integration via MCP: To further enhance the value proposition, taxonomy providers should consider surfacing their capabilities using the Model Context Protocol (MCP). MCP allows for standardized communication between different AI agents and systems, enabling seamless integration and interoperability. By making their taxonomies accessible via MCP, providers can ensure that agent builders can easily incorporate their data into their workflows, reducing friction and accelerating development.

3. Capitalizing on Existing Expertise as an Established Player:

Market Advantage: Emphasize that established taxonomy companies have a significant advantage due to their existing expertise, data assets, and client relationships. This position allows them to quickly adapt to the agent-driven market.

Economic Efficiency: Highlight the cost-effectiveness of using established taxonomy providers compared to building in-house solutions. Businesses looking to deploy agents quickly will likely prefer to partner with existing experts.

By focusing on the demand for content generation driven by the rise of intelligent agents and by targeting partnerships with agent-building organizations, taxonomy companies can position themselves for significant growth and success in this evolving market.

Why This Matters to You

We rely on AI more and more every day. From getting quick answers to complex research, we expect AI to provide us with accurate and reliable information. But what happens when the volume of information becomes overwhelming? What happens when AI systems need to sift through massive amounts of data to make critical decisions?

That's where organized data becomes vital. Imagine AI as a powerful detective tasked with solving a complex case. Without a well-organized case file (a robust taxonomy), the detective might get lost in a sea of clues, missing crucial details or drawing the wrong conclusions. But with a meticulously organized file, the detective can:

  • Quickly Identify Key Evidence: AI can pinpoint the most relevant and reliable information, even in a sea of data.
  • Connect the Dots: AI can understand the complex relationships between different pieces of information, revealing hidden patterns and insights.
  • Ensure a Clear Narrative: AI can present a coherent and accurate picture of the situation, avoiding confusion or misinterpretation.

In essence, the better the data is organized, the more effectively AI can serve as a reliable source of truth. It's about ensuring that AI doesn't just process information, but that it processes it in a way that promotes clarity, accuracy, and ultimately, a shared understanding of the world. This is why the role of taxonomies, ontologies, and metadata is so critical—they are the foundation for building AI systems that help us navigate an increasingly complex information landscape with confidence.

The Indispensable Role of Human Curation

While LLMs can be valuable tools in the taxonomy development process, they cannot fully replace human expertise (yet). Human curation is essential because taxonomies are ultimately designed for human consumption. Human curators can ensure that taxonomies are intuitive, user-friendly, and aligned with how people naturally search for and understand information. Human experts are needed not just for creating the taxonomy itself, but also for defining and maintaining the associated metadata and ontologies.

For example, imagine an LLM generating a taxonomy for a complex subject like "fine art." While it might group works by artist or period, a human curator would also consider factors like artistic movement, cultural significance, and thematic connections, creating a taxonomy that is more nuanced and useful for art historians, collectors, and enthusiasts.

alt text By Michelangelo, Public Domain, https://commons.wikimedia.org/w/index.php?curid=9097336

Developing a high-quality taxonomy often requires specialized knowledge of a particular subject area. Human experts can bring this knowledge to the process, ensuring that the taxonomy accurately reflects the complexities of the domain (for now).

Challenges and Opportunities

The rise of LLMs directly fuels the demand for sophisticated taxonomies. While LLMs can assist in generating content, taxonomies ensure that this content is organized, accessible, and contextually relevant. This dynamic creates both opportunities and challenges for taxonomy providers. The evolving nature of LLMs requires constant adaptation in taxonomy strategies, and the integration of metadata and ontologies becomes essential to maximize the utility of LLM-generated content. So, the expertise in developing and maintaining these taxonomies becomes a critical asset in the age of LLMs.

Enhanced Value Through Metadata and Ontologies

The value of taxonomies is significantly amplified when combined with robust metadata and ontologies. Metadata provides detailed descriptions and context, making taxonomies more searchable and understandable for LLMs. Ontologies, with their intricate relationships and defined properties, enable LLMs to grasp deeper contextual meanings and perform complex reasoning.

Metadata is data that describes other data. For example, the title, author, and publication date of a book are metadata. High-quality metadata, such as detailed descriptions, keywords, and classifications, makes taxonomies more easily searchable and understandable by both humans and machines, including LLMs. This rich descriptive information provides essential context that enhances the utility of the taxonomy.

Ontologies are related to taxonomies but go beyond simple hierarchical classification. While taxonomies primarily focus on organizing information into categories and subcategories, often representing "is-a" relationships (e.g., "A dog is a mammal"), ontologies provide a more detailed, formal, and expressive representation of knowledge. They define concepts, their properties, and the complex relationships between them. Ontologies answer questions like "What is this?", "What are its properties?", "How is it related to other things?", and "What can we infer from these relationships?"

Key Distinctions:

  • Relationship Types: Taxonomies mostly deal with hierarchical ("is-a") relationships. Ontologies handle many different types of relationships (e.g., causal, temporal, spatial, "part-of," "has-property").
  • Formality: Taxonomies can be informal and ad-hoc. Ontologies are more formal and often use standardized languages and logic (e.g., OWL - Web Ontology Language).
  • Expressiveness: Taxonomies are less expressive and can't represent complex rules or constraints. Ontologies are highly expressive and can represent complex knowledge and enable sophisticated reasoning.
  • Purpose: Taxonomies are primarily for organizing and categorizing. Ontologies are for representing knowledge, defining relationships, and enabling automated reasoning.

For instance, an ontology about products would not only categorize them (e.g., "electronics," "clothing") but also define properties like "manufacturer," "material," "weight," and "price," as well as relationships such as "is made of," "is sold by," and "is a component of." This rich, interconnected structure allows an LLM to understand not just the category of a product but also its attributes and how it relates to other products. This added layer of detail is what makes ontologies so valuable for LLMs, as they provide the deep, contextual understanding needed for complex reasoning and knowledge-based tasks. However, this level of detail also makes them more complex to develop and maintain, requiring specialized expertise and ongoing updates.

Therefore, companies that can integrate and provide these elements alongside taxonomies will offer a more compelling and valuable service in the LLM ecosystem. The combination of well-structured taxonomies, rich metadata, and detailed ontologies provides the necessary context and depth for LLMs to operate at their full potential.

Conclusion

The rise of LLMs is creating a classic supply and demand scenario. As more businesses adopt LLMs and techniques like RAG, the demand for structured data and the services of taxonomy providers will increase. However, it's crucial to recognize that the effectiveness of RAG hinges on high-quality data organization. Companies specializing in creating robust taxonomies, ontologies, and metadata are positioned to meet this demand by providing the essential foundation for successful RAG implementations. Their expertise ensures that LLMs and RAG systems can retrieve and utilize information effectively, making their services increasingly valuable for organizations looking to take advantage of LLM-generated content.

The AI-Driven Transformation of Software Development

1. Introduction: The Seismic Shift in Software Development

The software development landscape is undergoing a seismic shift, driven by the rapid advancement of artificial intelligence. This transformation transcends simple automation; it fundamentally alters how software is created, acquired, and utilized, leading to a re-evaluation of the traditional 'build versus buy' calculus. The pace of this transformation is likely to accelerate, making it crucial for businesses and individuals to stay adaptable and informed.

2. The Rise of AI-Powered Development Tools

For decades, the software industry has been shaped by a tension between bespoke, custom-built solutions and readily available commercial products. The complexity and cost associated with developing software tailored to specific needs often pushed businesses towards purchasing off-the-shelf solutions, even if those solutions weren't a perfect fit. This gave rise to the dominance of large software vendors and the Software-as-a-Service (SaaS) model. However, AI is poised to disrupt this paradigm.

Introduction to AI-Powered Automation

Large Language Models (LLMs) are revolutionizing software development by understanding natural language instructions and generating code snippets, functions, or even entire modules. Imagine describing a software feature in plain language and having an AI produce the initial code. Many are already using tools like ChatGPT in this way, coaching the AI, suggesting revisions, and identifying improvements before testing the output.

This is 'vibe coding,' where senior engineers guide LLMs with high-level intent rather than writing every line of code. While this provides a significant productivity boost—say, a 5x improvement—the true transformative potential lies in a one-to-many dynamic, where a single expert can exponentially amplify their impact by managing numerous AI agents simultaneously, each focused on different project aspects.

Expanding AI Applications in Development

Additionally, AI is being used for code review tools that can automatically identify potential issues and suggest improvements, and specific AI platforms offered by cloud providers like AWS CodeWhisperer and Google Cloud's AI Platform are providing comprehensive AI-driven development environments. AI is being used for AI-assisted testing and debugging, identifying potential bugs, suggesting fixes, and automating test cases.

Composable Architectures and Orchestration

Beyond code completion and generation, AI tools are also facilitating the development of reusable components and services. This move toward composable architectures allows developers to break down complex tasks into smaller, modular units. These units, powered by AI, can then be easily assembled and orchestrated to create larger applications, increasing efficiency and flexibility. Model Context Protocol (MCP) could play a role in standardizing the discovery and invocation of these services.

Furthermore, LLM workflow orchestration is also becoming more prevalent, where AI models can manage and coordinate the execution of these modular services. This allows for dynamic and adaptable workflows that can be quickly changed or updated as needed.

Human Role and Importance

However, it's crucial to recognize that AI is a tool. Humans will still be needed to guide its development, provide creative direction, and critically evaluate the AI-generated outputs. Human problem-solving skills and domain expertise remain essential for ensuring software quality and effectiveness.

Impact on Productivity and Innovation

These tools are not just incremental improvements; they have the potential to dramatically increase developer productivity, potentially enabling the same output with half the staff or even leading to a fivefold increase in efficiency in the near term, lower the barrier to entry for software creation, and enable the fast iteration of new features.

Impact on Offshoring

Furthermore, AI tools have the potential to level the playing field for offshore development teams. Traditionally, challenges such as time zone differences, communication barriers, and perceived differences in skill level have sometimes put offshore teams at a disadvantage. However, AI-powered development tools can mitigate these challenges:

  • Enhanced Productivity and Efficiency: AI tools can automate many tasks, allowing offshore teams to deliver faster and more efficiently, overcoming potential time zone delays.
  • Improved Code Quality and Consistency: AI-assisted code generation, review, and testing tools can ensure high code quality and consistency, regardless of the team's location.
  • Reduced Communication Barriers: AI-powered translation and documentation tools can facilitate clearer communication and knowledge sharing.
  • Access to Cutting-Edge Technology: With cloud-based AI tools, offshore teams can access the same advanced technology as onshore teams, eliminating the need for expensive local infrastructure.
  • Focus on Specialization: Offshore teams can specialize in specific AI-related tasks, such as AI model training, data annotation, or AI-driven testing, becoming highly competitive in these areas.

By embracing AI tools, offshore teams can overcome traditional barriers and compete on an equal footing with onshore teams, offering high-quality software development services at potentially lower costs. This could lead to a more globalized and competitive software development landscape.

3. The Explosion of New Software and Features

This evolution is leading to an explosion of new software products and features. Individuals and small teams can now bring their ideas to life with unprecedented speed and efficiency. This is made possible by AI tools that can quickly translate high-level descriptions into working code, allowing for quicker prototyping and development cycles.

Crucial to the effectiveness of these AI tools is the quality of their training data. High-quality, diverse datasets enable AI models to generate more accurate and robust code. This is particularly impactful in niche markets, where highly specialized software solutions, previously uneconomical to develop, are now becoming viable.

For instance, AI could revolutionize enterprise applications with greater automation and integration capabilities, lead to more personalized and intuitive consumer apps, accelerate scientific discoveries by automating data analysis and simulations, or make embedded systems more intelligent and adaptable.

Furthermore, AI can analyze user data to identify areas for improvement and drive innovation, making software more responsive to user needs. While AI automates many tasks, human creativity and critical thinking are still vital for defining the vision and goals of software projects.

It's important to consider the potential environmental impact of this increased software development, including the energy consumption of training and running AI models. However, AI-driven software also offers opportunities for more efficient resource management and sustainability in other sectors, such as optimizing supply chains or reducing energy waste.

Software will evolve at an unprecedented pace, with AI facilitating fast feature iteration, updates, and highly personalized user experiences. This surge in productivity will likely lead to an explosion of new software products, features, and niche applications, democratizing software creation and lowering the barrier to entry.

4. The Transformation of the Commercial Software Market

This evolution is reshaping the commercial software market. The proliferation of high-quality, AI-enhanced open-source alternatives is putting significant pressure on proprietary vendors. As companies find they can achieve their software needs through internal development or by leveraging robust open-source solutions, they are becoming more price-sensitive and demanding greater value from commercial offerings.

This is forcing vendors to innovate not only in terms of features but also in their business models, with a greater emphasis on value-added services such as consulting, support, and integration expertise. Strategic partnerships and collaboration with open-source communities will also become crucial for commercial vendors to remain competitive.

Commercial software vendors will need to adapt to this shift by offering their functionalities as discoverable services via protocols like MCP. Instead of selling large, complex products, they might provide specialized services that can be easily integrated into other applications. This could lead to new business models centered around providing best-in-class, composable AI capabilities.

Specifically, this shift is leading to changes in priorities and value perceptions. Commercial software vendors will likely need to shift their focus towards providing value-added services such as consulting, support, and integration expertise as open-source alternatives become more competitive. Companies may place a greater emphasis on software that can be easily customized and integrated with their existing systems, potentially leading to a demand for more flexible and modular solutions.

Furthermore, commercial vendors may need to explore strategic partnerships and collaborations with open-source communities to remain competitive and utilize the collective intelligence of the open-source ecosystem.

Overall, AI-driven development has the potential to transform the software landscape, creating a more level playing field for open-source projects and putting significant pressure on the traditional commercial software market. Companies will likely need to adapt their strategies and offerings to remain competitive in this evolving environment.

5. The Impact on the Open-Source Ecosystem

The open-source ecosystem is experiencing a significant transformation driven by AI. AI-powered tools are not only lowering the barriers to contribution, making it easier for developers to participate and contribute, but they are also fundamentally changing the competitive landscape.

Specifically, AI fuels the creation of more robust, feature-rich, and well-maintained open-source software, making these projects even more viable alternatives to commercial offerings. Businesses, especially those sensitive to cost, will have more compelling free options to consider. This acceleration is leading to faster feature parity, where AI could enable open-source projects to rapidly catch up to or even surpass the feature sets of commercial software in certain domains, further reducing the perceived value proposition of paid solutions.

Moreover, the ability for companies to customize open-source software using AI tools could eliminate the need for costly customization services offered by commercial vendors, potentially resulting in customization at zero cost. The agility and flexibility of open-source development, aided by AI, enable quick innovation and experimentation, allowing companies to try new features and technologies more quickly and potentially reducing their reliance on proprietary software that might not be able to keep pace.

AI tools can also help expose open-source components as discoverable services, making them even more accessible and reusable. This can further accelerate the development and adoption of open-source software, as companies can easily integrate these services into their own applications.

Furthermore, the vibrant and collaborative nature of open-source communities, combined with AI tools, provides companies with access to a vast pool of expertise and support at no additional cost. This is accelerating the development cycle, improving code quality, and fostering an even more collaborative and innovative environment. As open-source projects become more mature and feature-rich, they present an increasingly compelling alternative to commercial software, further fueling the shift away from traditional proprietary solutions.

6. The Changing "Build Versus Buy" Calculus

Ultimately, the rise of AI in software development is driving a fundamental shift in the "build versus buy" calculus. The rise of composable architectures means that 'building' now often entails assembling and orchestrating existing services, rather than developing everything from scratch. This dramatically lowers the barrier to entry and makes building tailored solutions even more cost-effective.

Companies are finding that building their own tailored solutions, often on cloud infrastructure, is becoming increasingly cost-effective and strategically advantageous. The ability for companies to customize open-source software using AI could eliminate the need for costly customization services offered by commercial vendors.

Innovation and experimentation in open-source, aided by AI, could further reduce reliance on proprietary software. Robotic Process Automation (RPA) bots can also be exposed as services via MCP, allowing companies to integrate automated tasks into their workflows more easily. This further enhances the 'build' option, as businesses can employ pre-built RPA services to automate repetitive processes.

7. Cloud vs. On-Premise: A Re-evaluation

The potential for AI-driven, easier on-premise app development could indeed have significant implications for the cloud versus on-premise landscape, potentially leading to a shift in reliance on big cloud applications like Salesforce.

There's potential for reduced reliance on big cloud apps. If AI tools drastically simplify and accelerate the development of custom on-premise applications, companies that previously opted for cloud solutions due to the complexity and cost of in-house development might reconsider. They could build tailored solutions that precisely meet their unique needs without the ongoing subscription costs and potential vendor lock-in associated with large cloud platforms.

Furthermore, for organizations with strict data sovereignty requirements, regulatory constraints, or internal policies favoring on-premise control, the ability to easily build and maintain their own applications could be a major advantage. They could retain complete control over their data and infrastructure, addressing concerns that might have pushed them towards cloud solutions despite these preferences.

While cloud platforms offer extensive customization, truly bespoke requirements or deep integration with legacy on-premise systems can sometimes be challenging or costly to achieve. AI-powered development could empower companies to build on-premise applications that seamlessly integrate with their existing infrastructure and are precisely tailored to their workflows.

Composable architectures can also make on-premise development more manageable. Instead of building large, monolithic applications, companies can assemble smaller, more manageable services. This can reduce the complexity of on-premise development and make it a more viable option.

Additionally, while the initial investment in on-premise infrastructure and development might still be significant, the elimination of recurring subscription fees for large cloud platforms could lead to lower total cost of ownership (TCO) over the long term, especially for organizations with stable and predictable needs.

Finally, some organizations have security concerns related to storing sensitive data in the cloud, even with robust security measures in place. The ability to develop and host applications on their own infrastructure might offer a greater sense of control and potentially address these concerns, even if the actual security posture depends heavily on their internal capabilities.

However, several factors might limit the shift away from big cloud apps:

The "As-a-Service" Value Proposition

Cloud platforms like Salesforce offer more than just the application itself. They provide a comprehensive suite of services, including infrastructure management, scalability, security updates, platform maintenance, and often a rich ecosystem of integrations and third-party apps. Building and maintaining all of this in-house, even with AI assistance, could still be a significant undertaking.

Moreover, major cloud vendors invest heavily in research and development, constantly adding new features and capabilities, often leveraging cutting-edge AI themselves. This pace of innovation in the cloud might be difficult for on-premise development, even with AI tools, to keep pace with.

Cloud platforms are inherently designed for scalability and elasticity, allowing businesses to easily adjust resources based on demand. Replicating this level of flexibility on-premise can be complex and expensive. Many companies prefer to focus on their core business activities rather than managing IT infrastructure and application development, even if AI makes it easier; the "as-a-service" model offloads this burden.

Large cloud platforms often have vibrant ecosystems of developers, partners, and a wealth of documentation and community support. Building an equivalent internal ecosystem for on-premise development could be challenging. Some advanced features, particularly those leveraging large-scale data analytics and AI capabilities offered by the cloud providers themselves, might be difficult or impossible to replicate effectively on-premise.

Cloud providers might also shift towards offering more granular, composable services that can be easily integrated into various applications. This would allow companies to leverage the cloud's scalability and infrastructure while still maintaining flexibility and control over their applications.

Therefore, a more likely scenario might be the rise of hybrid approaches, where companies use AI to build custom on-premise applications for specific, sensitive, or highly customized needs, while still relying on cloud platforms for other functions like CRM, marketing automation, and general productivity tools.

While the advent of AI tools that simplify on-premise application development could certainly empower more companies to build their own solutions and potentially reduce their reliance on monolithic cloud applications like Salesforce, a complete exodus is unlikely. The value proposition of cloud platforms extends beyond just the software itself to encompass infrastructure management, scalability, innovation, and ecosystem.

Companies will likely weigh the benefits of greater control and customization offered by on-premise solutions against the convenience, scalability, and breadth of services provided by the cloud. We might see a more fragmented landscape where companies strategically choose the deployment model that best fits their specific needs and capabilities.

8. The AI-Driven Software Revolution: A Summary

The integration of advanced AI into software development is poised to trigger a profound shift, fundamentally altering how software is created, acquired, and utilized. This shift is characterized by:

1. Exponential Increase in Productivity and Innovation:

AI as a Force Multiplier: AI tools are drastically increasing developer productivity, potentially enabling the same output with half the staff or even leading to a fivefold increase in efficiency in the near term.

Cambrian Explosion of Software: This surge in productivity will likely lead to an explosion of new software products, features, and niche applications, democratizing software creation and lowering the barrier to entry.

Rapid Iteration and Personalization: Software will evolve at an unprecedented pace, with AI facilitating fast feature iteration, updates, and highly personalized user experiences. This will often involve complex LLM workflow orchestration to manage and coordinate the various AI-driven processes.

This impact will be felt across various types of software, from enterprise solutions to consumer apps, scientific tools, and embedded systems. The effectiveness of these AI tools relies heavily on the quality of their training data, and the ability to analyze user data will drive further innovation and personalization.

We must also consider the sustainability implications, including the energy consumption of AI models and the potential for AI-driven software to promote resource efficiency in other sectors. These changes are not static; they are part of a dynamic and rapidly evolving landscape. Tools like GitHub Copilot and AWS CodeWhisperer are already demonstrating the power of AI in modern development workflows.

2. Transformation of the Software Development Landscape:

Evolving Roles: The traditional role of a "coder" will diminish, with remaining developers focusing on AI prompt engineering, system architecture, including the design and management of complex LLM workflow orchestration, integration, service orchestration, MCP management, quality assurance, and ethical considerations.

This shift is particularly evident in the rise of vibe coding. More significantly, we're moving towards a one-to-many model where a single subject matter expert (SME) or senior engineer will manage and direct many LLM coding agents, each working on different parts of a project. This orchestration of AI agents will dramatically amplify the impact of senior engineers, allowing them to oversee and guide complex projects with unprecedented efficiency.

AI-Native Companies: New companies built around AI-driven development processes will emerge, potentially disrupting established software giants.

Democratization of Creation: Individuals in non-technical roles will become "citizen developers," creating and customizing software with AI assistance.

3. Broader Economic and Societal Impacts:

Automation Across Industries: The ease of creating custom software will accelerate automation in all sectors, leading to increased productivity but also potential job displacement.

Lower Software Costs: Development cost reductions will translate to lower software prices, making powerful tools more accessible.

New Business Models: New ways to monetize software will emerge, such as LLM features, data analytics, integration services, and specialized composable services offered via MCP.

Workforce Transformation: Educational institutions will need to adapt to train a workforce for skills like AI ethics, prompt engineering, and high-level system design.

Ethical and Security Concerns: Increased reliance on AI raises ethical concerns about bias, privacy, and security vulnerabilities. This includes the challenges of handling sensitive data when using AI tools.

4. Implications for Purchasing Software Today:

Short-Term vs. Long-Term: Businesses must balance immediate needs with the potential for cheaper and better AI-driven alternatives in the future.

Flexibility and Scalability: Prioritizing flexible, scalable, and cloud-based solutions is crucial.

Avoiding Lock-In: Companies should be cautious about long-term contracts and proprietary solutions that might become outdated quickly.

5. Google Firebase Studio as an Example:

AI-Powered Development: Firebase Studio's integration of Gemini and AI agents for prototyping, feature development, and code assistance exemplifies the trend towards AI-driven development environments.

Rapid Prototyping and Iteration: The ability to create functional prototypes from prompts and iterate quickly with AI support validates the potential for an explosion of new software offerings.

In essence, the AI-driven software revolution represents a fundamental shift in the "build versus buy" calculus, empowering businesses and individuals to create tailored solutions more efficiently and affordably. While challenges exist, the long-term trend points towards a more open, flexible, and dynamic software ecosystem. It's important to remember that AI is a tool that amplifies human capabilities, and human ingenuity will remain at the core of software innovation.

9. Conclusion: A More Open and Dynamic Software Ecosystem

In conclusion, the advancements in AI are ushering in an era of unprecedented change in software development. This transformation promises to democratize software creation, accelerate innovation, and empower businesses to build highly customized solutions. While challenges remain, the long-term trend suggests a move towards a more open, composable, flexible, and user-centric software ecosystem, increasingly driven by discoverable services. Furthermore, the pace of these changes is likely to accelerate, making adaptability and continuous learning crucial for both businesses and individuals.

  • in prompt
  • 8 min read

2025 01 31 prompt guide

Background

While this post is mainly intended for users who want to write custom prompts for agentic interpretation of various trading indicators, fundamental and macroeconomic analyses, and other relevant inputs, the information contained within this post can be informative for other varieties of LLM prompting.

General Notes

I’ve found that there is a generally good ‘order’ to doing prompts.

  1. General background information that needs to be included somewhere
  2. Specific data and analysis unique to this specific instance of prediction
  3. Reminders and anti-hallucination hints

The reasoning for this is that the most recent tokens in general have the most impact on the output when there is a conflict. So, we want things that are preventing hallucinations and critical operational information to be the most recent tokens, otherwise they could be outweighed by less relevant information that could trigger a hallucination or other response malformation.

For background information, a lot of it could be viewed as ‘reminders’. Most of this information should exist somewhere in the training data/saved weights. We just want to make sure that the critical stuff is at the ‘forefront of the mind’ so to speak. The other reason to add this information is to remove any ambiguity that might exist in the prompt. I’ll go more into specific examples later.

For specific data and analysis, try not to be too wordy/verbose. The last thing you want to have is the LLM clinging on to the linguistic flavor that you included. THESE MODELS ARE NOT PEOPLE and you would do very well to remember that. You can have some oddly formatted or abrupt sentences as long as all of the information included within is accurate and correctly formed. Words like “Please” and “Thank You” are just going to be extra tokens adding spacing between related tokens. Sometimes these agents can get a little lost and think that they are having a conversation with you. That is not what you want - all the information it outputs should be entirely mission focused. If your agent is saying things like “I’m sorry but” or “Certainly, I can” then you likely need to clamp down on your prompt. These outputs are being fed into other automated systems, which don’t need that information, and these kinds of outputs can even cause errors depending on the system consuming the data.

Additionally, it is important to remember that LLMs ARE NOT A FORM OF ARTIFICIAL GENERAL INTELLIGENCE. LLMs are extremely advanced token prediction models. There are actual benchmarks and tests for AGI that LLMs are not even close to meeting, and the actual experts in the field are unanimous in their agreement that LLMs are not AGI. No matter what you hear from a CEO or tech influencer, remember this information. Chain of thought or whatever new paradigm being pushed is not the model actually thinking. These models are not capable of actual thought and they are not sentient or sapient. Wherever you are trying to figure out if an LLM can solve a problem or why it is failing to solve a problem, you should ask yourself if a highly advanced token prediction system can actually solve the problem. If the answer is no, then the LLM is not going to be able to do it. This guide was written in 2025, and while there will likely be amazing advances in the future, I can say with confidence that LLMs are not anywhere close to breaking the boundaries of AGI. If this AGI boundary is actually broken, then this prompting guide will be wholly obsolete and none of the information contained within will be relevant. So, if you are referencing this guide to solve a problem then this section is still completely accurate. The entirety of this document is written with the understanding of this section as being an absolute, non-arguable axiom, and if it was not true then this document would be worthless. While some readers might view this section as strange/obvious/irrelevant, I can assure you that some people using this guide to write prompts absolutely require this information.

For the following specific examples and analysis, the prompt is formatted with an underline, while my analysis and suggestions are inserted between specific sections of the prompt. This is so that the information is better localized for your reading. If you want to know what the original prompt was, just read the underlined sections, skipping any text that is not underlined.

You are running an analysis on the output of an advanced deep learning model that is making a buy or don't buy decision for this stock.

We start out here with the basic background information that sets the stage for what the model is doing and why. In a standard human to human conversation this information makes sense to be the first part of the conversation. As such, it is likely that the training data for the model would follow this kind of linguistic flow. Furthermore, it is pretty unlikely that hallucinations or any other information included within this prompt would take the model off task here. Most of the hallucinations I see are misattributing data points between metrics or adding in additional information. If you end up having an issue with this (likely due to having an extremely long prompt) then a follow up reminder at the end is your best bet. I’d still recommend leaving in this sentence at the start in that event, due to the reasons described above, as well as the issue being severe enough to even warrant additional action.

The model gives a value between 0 and 1, with 0 being a low confidence for a buy and 1 being a high confidence for a buy. This model is shown to have a 60% accuracy on back test data. The most recent value from the model is the most important indicator, as older values are for times in the past. This historical change in the model's output is included for informative purposes, as an increasing value indicates that market conditions are increasingly favorable for a buy. When making your analysis, don't get confused by the min, max, and mean values provided. These are to be considered in comparison to the most recent value, which is the value you should be considering. In general, a value above a 0.5 is considered a buy signal and a value below 0.5 is considered to not be a buy signal.

Now we have our information on how to actually interpret the results of the model. My first critique of this section (which for the record I wrote) is that the 0 and 1 explanations are ambiguous and likely to cause issues. This observation is backed up by empirical evidence as well, as I’ve seen this model list values of 0.4 as “high confidence”. A better formulation might be:

“0 is the lowest confidence in being a good buy, and 1 is the highest confidence for a good buy. Values below 0.5 are evaluated by training and validation algorithms as not containing a buy signal, while values above 0.5 are considered to be buy signals. By this formulation, taking all values above 0.5 as being a buy gives a 60% accuracy on back test data.”

By doing this we are also placing the information about the 0.5 boundary closer to the information on how to evaluate the range between 0 and 1, which is likely to give us better results. Furthermore, as this is some of the most critical information on how to interpret the model, I would recommend moving it lower in the prompt, especially if the agent is making mistakes in analyzing if individual predictions are good buys or not good buys.

We also have in this section the disclaimer about the historical values that we include. This was originally included as the agent was thinking that some of those historical and aggregate values represented the actual prediction of the model. We included that data to allow the agent to look for outliers and trends. If we see a model sitting around 0.35 for a period of time, and then our most recent value is a 0.49, one might conclude that market conditions had drastically changed, and it might be a good time to buy even though we haven’t crossed the 0.5 threshold. On the other hand, if we’ve been oscillating between 0.48 and 0.52, a value of 0.49 is much less significant. All of this information would be lost if we simply provided the most recent value, as both situations would be showing a 0.49. You might also wonder why we don’t just build out a mathematical model that makes this decision for us. While this is often a good approach when dealing with LLM models, this situation is one filled with ambiguity and judgement calls. What if rather than a value of 0.35, it had been a value of 0.38/0.4/0.45? Where does the cutoff exist? One could use fuzzy logic or a similar approach, but one could also just leave this decision up to the agent. If something has an objective, mathematical, modelable, right/wrong answer, then you should use that mathematical model to give you an answer instead of the LLM agent. If something needs a judgement call and you cannot determine the right answer until after a prediction is made, then the LLM agent is more appropriate.

Analysis of ensemble_output: min=0.3541967868804931, max=0.5319923162460327, mean=0.40127307176589966. Regression coef=[-0.00171258], score=0.07836451998035432, direction=decreasing. Latest value=0.4099920988082886.

Now we get to the actual information for this specific prediction. Everything up to this point is going to be identical for every single prediction we make. The first thing of note is all those significant digits on all of the numeric values. For Llama 3.2, the string "min=0.3541967868804931" has 11 tokens, while “min=0.354” has 5 tokens. Are all of those extra digits actually meaningful for our prediction? While you might be asking yourself if this matters, consider that "min=0.3541967868804931, max=0.5319923162460327, mean=0.40127307176589966" and "Wow! we have a min value of 0.354, a max value of 0.532, and a mean (not angry mean tho) value of 0.401" both have 35 tokens using Llama 3.2. If you saw that second string, wouldn’t you go in and trim out all that useless extra text? None of those extra tokens provide any meaningful value to our analysis, and only serve to dilute and separate information. Meanwhile, "min of 0.354, max of 0.532, mean of 0.401" has 18 tokens, with a reduction of almost 50%.

What about “Regression coef=[-0.00171258]”? For just 1 more token, we could instead have "Regression coef of -0.002, indicating a downward trend". Which one of those strings contains more useful information? And with all those savings in token count, we could turn "score=0.07836451998035432" (with a token count of 12) into "and an r squared score of 0.078, indicating that this data is not very linear in nature" (with a token count of 21). What a difference in interpretability and information density 9 tokens can make!

That's all for now.

  • in deepseek
  • 1 min read

2025 01 30 deepseek

Just a quick heads up. As early as Monday afternoon the Deepseek-r1 reasoning model was available to all of our agent technology licensees.

alt text

2025 01 28 deepseek plausible

DeepSeek's Key Innovations: A Brief Analysis

1. FP8 Mixed Precision Training
  • What it does: Reduces memory and compute requirements by representing numbers with 8-bit precision instead of 16-bit (FP16) or 32-bit (FP32).
  • Impact: FP8 mixed precision training nearly doubles throughput on H800 GPUs for tensor operations like matrix multiplications, which are central to transformer workloads. The Hopper architecture’s Tensor Cores are designed for FP8 precision, making it highly effective for large-scale deep learning tasks that require both computational efficiency and high throughput.
  • Estimated Gain: ~1.8x performance boost, critical for achieving high token throughput.
2. MoE Architecture
  • What it does: Activates only 37B parameters per token out of 671B, significantly reducing the compute cost for forward and backward passes.
  • Impact: Sparse activation significantly reduces computational overhead without compromising representational power.
  • Estimated Gain: Estimated Gain: 5–10x improvement in compute efficiency compared to dense architectures.
3. Auxiliary-Loss-Free Load Balancing
  • What it does: Eliminates the auxiliary-loss typically used to balance expert activation in MoE, reducing inefficiencies and avoiding performance degradation.
  • Impact: Improves token processing efficiency without wasting GPU cycles on balancing overhead.
  • Estimated Gain: ~5–10% boost in efficiency, depending on the prior impact of auxiliary losses.
4 Multi-Token Prediction (MTP):
  • What it does: Predicts two tokens per forward pass instead of one, reducing the number of forward passes required for training and decoding. The speculative decoding framework validates the second token, with an acceptance rate of 85–90%.
  • Impact:
    • Fewer forward passes accelerate training, improving throughput.
    • The high acceptance rate ensures minimal overhead from corrections.
  • Estimated Gain: ~1.8x improvement in token processing efficiency, depending on model configuration and workload.
5. Communication-Compute Overlap
  • What it does: Optimizes distributed training by overlapping inter-GPU communication with computation, addressing a common bottleneck in large-scale MoE systems.
  • Impact: Removes inefficiencies that typically reduce utilization in cross-node setups.
  • Estimated Gain: Allows near-100% utilization of GPU capacity during training.

Hardware Considerations

DeepSeek trained its model on a cluster of 2,048 H800 GPUs, leveraging Nvidia's Hopper architecture. These GPUs are designed to excel at tasks like matrix multiplications and sparse attention, particularly when using FP8 mixed precision. While the H800 has lower interconnect bandwidth compared to the H100 due to export regulations, its computational efficiency remains strong for the kinds of workloads needed in large-scale AI training.

Token Throughput Calculation

Using their stated figures, let’s verify whether the throughput aligns with the claimed GPU-hour budget.

  1. Throughput per GPU-Hour:

    • Tokens Processed: 14.8 trillion tokens.
    • GPU-Hours Used: 2.664 million.
    • Tokens per GPU-Hour:

\(\frac{14.8 \, \text{trillion tokens}}{2.664 \, \text{million GPU-hours}} = 5.56 \, \text{million tokens per GPU-hour}.\)

  1. Cluster Throughput:
    • GPUs Used: 2,048.
    • Tokens Processed per Hour:

\(2,048 \, \text{GPUs} \times 5.56 \, \text{million tokens per GPU-hour} = 11.38 \, \text{billion tokens per hour}.\)

  1. Time to Process 14.8T Tokens:
    • Total Time:

\(\frac{14.8 \, \text{trillion tokens}}{11.38 \, \text{billion tokens per hour}} = 1,300 \, \text{hours (or ~54 days)}.\)

This aligns with their claim of completing pretraining in less than two months.

Conclusion

DeepSeek-V3’s claim of processing 14.8T tokens in 2.664M GPU-hours is plausible. The numbers are internally consistent, and the described techniques align with established principles of efficient large-scale training. While reproduction by other labs will provide final confirmation, the absence of red flags suggests that DeepSeek's reported achievements are feasible.

For more details on DeepSeek-V3 and its training methodology, refer to the technical report on arXiv.

Qualifications: Through my work on the LIT platform, I’ve developed tools that enable data scientists to efficiently design, train, and deploy deep learning models, including advanced workflows for LLMs. Prior to that, I spent 8 years providing professional services in deep learning, building custom AI solutions across diverse industries. In support of that work I’ve read and analyzed hundreds of technical reports and academic papers. My expertise lies in building tooling, pipelines, and integrations for both predictive and generative AI, supported by a strong foundation in deep learning and software engineering.