The Power of Code Generation: Programming Languages to AI Assistants

Imagine you are tasked to write 417,823 lines of code in 6 different languages. What if there was a way to slash that development effort significantly with just a fraction of the effort? Introducing code generation, where a few high-level instructions can do the work for you. In this blog post, we'll dive into the superpower of code generation, uncovering its presence from early programming to modern-day AI assistants.

Code generation is ubiquitous in software engineering, it exists in literally every part of the stack from programming languages to the websites we browse. So why is code generation everywhere? Because code generation empowers developers. By automating tasks and creating simpler abstractions, code generation makes developers more productive and less error-prone.

At C3.ai, I worked on a tool that leverages code generation to build enterprise AI applications. At the same time, I became deeply immersed in the art of code generation by building the tooling myself. Since then, I enjoy learning about code-generation technologies across the entire software stack from programming languages to AI assistants.

Mindmap of code generation topics covered in this post

Why Generate Code?

Code generation is the automated process of producing code from a high-level representation. Code generation tools parse input data to produce target code in programming languages. The most notable high-level representation is programming languages.

Code generators transform high level representations to code

When the first electrical computers were created, physical switches were used to program computers. Physical switches are extremely error-prone and time-consuming to operate. So with the invention of the CPU and instruction sets, the first software form of a programming language was invented, assembly.

hello_world.X86

_19

section .data

_19

hello db 'Hello, World!',0xA

_19

hello_len equ $ - hello

_19

_19

section .text

_19

global _start

_19

_19

_start:

_19

; syscall to print the message

_19

mov eax, 4 ; sys_write syscall number

_19

mov ebx, 1 ; file descriptor (stdout)

_19

mov ecx, hello ; pointer to the message

_19

mov edx, hello_len ; message length

_19

int 0x80

_19

_19

; syscall to exit the program

_19

mov eax, 1 ; sys_exit syscall number

_19

xor ebx, ebx ; exit status (0 = success)

_19

int 0x80

Example of a "Hello, World!" program in Intel x86 assembly language.

Assembly is a linear series of instructions for the computer to execute. Assembly is used to generate the lowest level of instructions for computers, machine code.

Can you believe this?

The classic amusement park simulator RollerCoaster Tycoon was written using assembly.

But still, assembly code is tedious and error-prone to write so programming languages that generate to assembly were invented. Here is the same "Hello, World!" program written in popular programming languages.

C++

JavaScript

Python

Go

Java

_10

#include <iostream>

_10

_10

int main() {

_10

std::cout << "Hello, World!" << std::endl;

_10

return 0;

_10

}

After comparing the "Hello, World!" program in assembly to the ones written in higher-level programming languages, it's clear that code generation empowers developers to be 100x more productive. If you think about it, software engineering is simply operating code generators to produce performant machine code 🤯.

Compilers are the code generators that power programming languages. Compilers turn a source language into a target language. In practice, most computer languages target assembly code in instruction sets such as x86 and ARM, but others like Java target their own instruction set. Some programming languages compile to other programming languages such as TypeScript to JavaScript.

Fun fact

Compilers themselves are typically written in the programming language itself. This concept is called bootstrapping.

Nowadays, compilers are so advanced that there is rarely a reason to write pure assembly code. In most cases, writing applications in a programming language leads to faster programs than writing programs in assembly due to amazing compiler infrastructures like GCC and LLVM.

Interesting Topic

Watch this fascinating video about branchless programming. The video covers an example of why in some cases you need to know exactly how programming languages work to beat the compiler.

Database Code Generation

One code-generation pattern that has found a lot of traction is in Object-Relational Mapping (ORM).

ORMs generates code used in application code that helps developers easily execute SQL queries

For context, databases define a domain-specific language that allows them to build a generic execution engine for querying data. Databases can then optimize the execution engine while still providing a flexible interface for querying data. Today the most common type of query language for databases is SQL (aka Structured query language). But SQL is tedious to read and write, especially if you have to write many read or write operations to a database in a single codebase.

query.sql

_10

SELECT id, name, age, department

_10

FROM employees

_10

WHERE age > 30

_10

ORDER BY name;

An example SQL query for employees over the age of 30

Enter Prisma, an extremely popular library with 33k stars on GitHub for building applications that interact with a database like PostgreSQL or MongoDB. Prisma is an ORM that leverages code generation to allow developers to easily use databases by generating database migration queries in SQL and type-safe clients to interact with the database.

Source: prisma.io

Take for example a TypeScript function that makes two simple queries. With Prisma, we can convert plain-text SQL statements to typed function calls and delete over 50% of the code.

The same function for querying a database in TypeScript using plain-text SQL statements (Left) and Prisma (Right)

Prisma's superior developer experience allows developers to move fast without breaking things. Unsurprisingly, Prisma has found itself as a fan favorite amongst Node.js developers, quickly beating out existing ORMs.

Despite only appearing in 2019, Prisma is rapidly beating out Sequelize, a much older ORM for Node.js (Source: star-history.com)

Other modern ORMs like EdgeDB recognize the benefits of code generation in delivering a great developer experience and are actively integrating code generation capabilities into their ORM.

Source: edgedb.com

By taking database interactions to a higher level, Prisma and EdgeDB hope to build an intelligent layer over SQL that can produce queries that would take a human hours or even days to craft. But in general, ORMs like Prisma and EdgeDB leverage code generation to fulfill an ongoing pursuit of more efficient, developer-friendly, and maintainable solutions for database interactions in modern software development.

Web Application Code Generation

Web application technologies have existed since the dawn of Web 2.0. In parallel, frameworks for building web applications have found traction amongst developers. This is because web frameworks abstract or automate the tedious code needed to build full-stack applications.

Ruby on Rails, which has been used to build giant tech companies such as Shopify and GitHub, is famous for tooling that automates the tedious work when building a full-stack application. Rails does this by providing a CLI for generating commonly used code while developing.

Source: rubyonrails.org

For example, if you wanted to create a blog as a Rails application, you would simply run:

Terminal

_10

rails new blog

The CLI will then set up all the boilerplate code to start developing your web application in Rails under the blog/ folder. The rails CLI also offers code generation for tests, database migrations, models, and more. Code generation with Rails saves developers hours of setup and maintenance work. Teams of developers also move faster with code generation by enforcing best practices within the generators themselves.

other examples

RedwoodJS, a full-stack framework built on top of modern web technologies like GraphQL, TypeScript, and Prisma, leverages code generation to generate UI components in the form of cells.

Source: redwoodjs.com

Django, the Python project powering products like Instagram, leverages code generation to generate database migrations.

Source: djangoproject.com

Another use case for code generation appears in the latest front-end framework for browsers, Svelte. Svelte disrupts front-end frameworks such as React or Vue by introducing a compiler that drastically improves the developer experience when building complex front-end applications.

Source: svelte.dev

The main benefits of the Svelte compiler are reduced bundle sizes, performant front-end code, and improved developer experience. For example, here is a simple web page written in Svelte that displays how many times a button was clicked.

index.svelte

_13

<script>

_13

let count = 0;

_13

_13

function increment() {

_13

count += 1;

_13

}

_13

</script>

_13

_13

<main>

_13

<h1>Reactivity in Svelte</h1>

_13

<p>Count: {count}</p>

_13

<button on:click={increment}>Increment</button>

_13

</main>

The same web page written in React has additional import statements and boilerplate code from setting up a functional component and state.

index.jsx

_19

import React, { useState } from 'react';

_19

_19

function ReactiveReact() {

_19

const [count, setCount] = useState(0);

_19

_19

function increment() {

_19

setCount(count + 1);

_19

}

_19

_19

return (

_19

<div>

_19

<h1>Reactivity in React</h1>

_19

<p>Count: {count}</p>

_19

<button onClick={increment}>Increment</button>

_19

</div>

_19

);

_19

}

_19

_19

export default ReactiveReact;

Boilerplate code that is not necessary when developing with Svelte

The simplicity of Svelte is made possible by its compiler. By adding a code generation layer at build time, Svelte can provide a clean interface for building front ends while maintaining reactivity and state management functionality.

The Svelte REPL showing the input file (left) and the compiled JavaScript output (right)

Web API Code Generation

Adjacent to web applications are Web APIs, which can be really complex. But code generation drastically simplifies API integrations by generating boilerplate code and client libraries. Network protocols such as GraphQL, gRPC, and OpenAPI not only streamline API development but also leverage code generation to create server-side scaffolding and client-side SDKs, enhancing API consistency and reducing complexity.

GraphQL has found a lot of traction when building front-end applications due to its self-documenting workflow and flexibility for front-ends to query for only necessary data. Developers can generate clients for a GraphQL API to remove any boilerplate code and enforce type safety. For example given the simple GraphQL schema for a User model and a getUser query.

schema.graphql

_10

type User {

_10

id: ID!

_10

name: String!

_10

email: String!

_10

}

_10

_10

type Query {

_10

getUser(id: ID!): User

_10

}

We can leverage graphql-code-generator to generate type-safe query hooks in React. Notice that we simply import the query and pass in the necessary variable, userId. No need to manually construct an HTTP request, serialize a request body or deserialize a response body.

_30

import React from 'react';

_30

import { useGetUserQuery } from '../generated/graphql';

_30

_30

const UserComponent = ({ userId }) => {

_30

const { loading, error, data } = useGetUserQuery({ variables: { userId } });

_30

_30

if (loading) {

_30

return <div>Loading...</div>;

_30

}

_30

_30

if (error) {

_30

return <div>Error: {error.message}</div>;

_30

}

_30

_30

const user = data?.getUser;

_30

_30

return (

_30

<div>

_30

<h2>User Information</h2>

_30

{user && (

_30

<div>

_30

<p>Name: {user.name}</p>

_30

<p>Email: {user.email}</p>

_30

</div>

_30

)}

_30

</div>

_30

);

_30

};

_30

_30

export default UserComponent;

The highlighted lines mark the use of generated code

For gRPC, communication happens between two backend systems so code is generated for a server-side language such as Python. Using the same User example in GraphQL, the .proto file for gRPC is the following.

user_service.proto

_17

syntax = "proto3";

_17

_17

package user;

_17

_17

service UserService {

_17

rpc GetUser(GetUserRequest) returns (UserResponse) {}

_17

}

_17

_17

message GetUserRequest {

_17

int32 user_id = 1;

_17

}

_17

_17

message UserResponse {

_17

int32 id = 1;

_17

string name = 2;

_17

string email = 3;

_17

}

Using grpcio-tools to generate a client in Python we can write the following code to communicate with the UserService.

make_request.py

_21

import grpc

_21

import user_service_pb2

_21

import user_service_pb2_grpc

_21

_21

def run_grpc_client():

_21

channel = grpc.insecure_channel('localhost:50051')

_21

client = user_service_pb2_grpc.UserServiceStub(channel)

_21

_21

try:

_21

user_id = 1

_21

request = user_service_pb2.GetUserRequest(user_id=user_id)

_21

response = client.GetUser(request)

_21

print(f"User ID: {response.id}")

_21

print(f"Name: {response.name}")

_21

print(f"Email: {response.email}")

_21

_21

except grpc.RpcError as e:

_21

print(f"Error: {e}")

_21

_21

if __name__ == '__main__':

_21

run_grpc_client()

The highlighted lines mark the use of generated code

In OpenAPI, communication is typically between a third-party developer and a public API such as Stripe. Using the same User example again we can define the following OpenAPI.

api.yaml

_32

openapi: 3.0.0

_32

info:

_32

title: Sample API

_32

version: 1.0.0

_32

paths:

_32

/users/{user_id}:

_32

get:

_32

summary: Get user by ID

_32

parameters:

_32

- name: user_id

_32

in: path

_32

required: true

_32

schema:

_32

type: integer

_32

responses:

_32

'200':

_32

description: Successful response

_32

content:

_32

application/json:

_32

schema:

_32

$ref: '#/components/schemas/User'

_32

components:

_32

schemas:

_32

User:

_32

type: object

_32

properties:

_32

id:

_32

type: integer

_32

name:

_32

type: string

_32

email:

_32

type: string

And use openapi-generator to generate a publishable client SDK in Python.

make_request.py

_20

from python_client.api_client import ApiClient

_20

from python_client.api.default_api import DefaultApi

_20

_20

def get_user_by_id(user_id):

_20

configuration = Configuration()

_20

_20

api_client = ApiClient(configuration)

_20

api_instance = DefaultApi(api_client)

_20

_20

try:

_20

response = api_instance.get_user_by_id(user_id)

_20

print(f"User ID: {response.id}")

_20

print(f"Name: {response.name}")

_20

print(f"Email: {response.email}")

_20

_20

except ApiException as e:

_20

print(f"Error: {e}")

_20

_20

if __name__ == '__main__':

_20

get_user_by_id(1) # Replace '1' with the desired user ID

The highlighted lines mark the use of generated code

In all three examples, generated client SDKs remove unnecessary code to construct an HTTP request or handle serialization and deserialization. Furthermore, any unexpected data errors are caught during development time. As the number of operations and data models grows, the same code generation tooling can be used to ensure consistency throughout API integrations.

When Not To Use Code Generation?

Like everything in engineering, there are tradeoffs. Even though code generation is powerful, generating code adds a layer of complexity that is not always warranted. In some cases, it does not make sense to generate code as it slows the development process or the same effect can be achieved more simply. Compilers are not simple; the LLVM has over 400,000 commits for a good reason. Compilers have to treat code as data which leads to wildly complex ASTs and data flow analysis.

An example of when code generation should not be used is when building a data validation library for TypeScript or Python. You could theoretically generate data validation code from a standard specification such as JSON Schema. But languages like TypeScript and Python already come prepared with the typing functionality. Instead, we should re-use the typing functionality to create our data validation library. Popular examples of non-code-generation-based data validation libraries include zod for TypeScript and pydantic for Python. No compilation step is necessary which means they can be imported at any time and used in any runtime.

But when process boundaries are crossed such as HTTP APIs, data validation is not possible to achieve without code generation; hence the use of code generation in network protocols like GraphQL, gRPC, and OpenAPI.

AI Assistants

The latest form of code generation looks completely different than previous generations. Instead of domain-specific toolings like how ORMs generate SQL, AI assistants define a whole new input and output model. In particular, products such as CodePilot and Tabnine live inside of a developer's IDE, generating contextual code from the currently open files.

Demonstration of AI-generated code inside the IDE (Source: github.com/features/copilot)

What do I mean by contextual code? This time, instead of providing a standard input format such as a programming language or API specification, the input format is simply the surrounding code in the open files. A suggested completion is predicted based on the context and offered as a tab-completion in the IDE. Since LLMs are probability-based models, its usefulness revolves entirely around prediction accuracy. A probability model works out great in IDEs because the generated code can easily be assessed by a human and there is virtually no risk when the suggestion is wrong.

The Cons of AI Assistants

As a Copilot power user, I can attest to its value today. But I can also attest to how Copilot introduces unwanted new complexity. But don't just take it from me, Microsoft even published a video guide on "Prompting with Copilot", teaching developers how to wrangle the complexity of prompting the AI while using Copilot. One tip from the video is to write an explanatory comment at the top of a file to give Copilot more context. For example, you could write the following comment at the top of a Python file.

main.py

_10

# Description: The file processes a JSON

_10

_10

def process_json(json):

_10

# ...

Putting more context in the file as a comment increases the accuracy of Copilot

Since everything is probability-based, providing enough context for the AI is crucial to improve its accuracy. But sometimes prompt engineering can take more time and effort than simply writing the intended code. And it doesn't help that AI is a black box, making experimentation a dark art.

When lots of context is required, which is often the case in larger codebases, the accuracy of the AI drops which can then lead to misleading or incorrect suggestions. Magic, a company building a software engineer inside your computer, recognizes this limitation and is on a mission to solve it.

Source: magic.dev

My Thoughts on AI

Over time, we have engineered ourselves into higher and higher levels of abstraction to build performant software. Mojo, a new programming language for writing performant low-level code in plain Python, is an interesting and recent example.

Source: modular.com/mojo

From low-level assembly to AI assistants, compilers and code generation has proven to be a major productivity multiplier for software engineers. And with the most recent advancements in AI, we find ourselves asking, "Is natural language the final form of programming?". If a rules-based engine can be outperformed by LLMs, why spend thousands of engineering hours building one?

I believe completion-based assistants have a strong fit in the software development lifecycle, but LLMs just don't cut it when you need 100% accuracy, which is a common requirement in software engineering. But that doesn't stop some companies from nobly going all-in on AI-based code generation for problems. But from prompt wrestling experience with GitHub Copilot, which arguably probably has the most refined UX and traction amongst developers, I only see developers leveraging AI code generation in problem spaces where inaccuracy is acceptable. That being said, I am I would happily delete our entire codebase if AI can adequately solve the SDK generation problem.

Looking Into the Future

Code generation is a testament to the remarkable power of automation in software. From its early days in assembly language to the modern AI assistants, code generation has continuously evolved, unlocking new levels of productivity and efficiency for developers.

As we move forward, code generation will continue to be a vital tool, aiding developers in their pursuit of faster, more maintainable, and error-free software solutions. Embracing the right balance between automation and human expertise will be crucial in harnessing the full potential of code generation while acknowledging its limitations.

Are you leveraging code generation? What exciting advancements will the next generation of code generation bring? Will AI assistants revolutionize software engineering?