We are three years into the current AI experiment and the primary interface remains a chat box.
It feels like a longer time, but the screen looks the same. We type questions into a text area and wait for answers. This approach works as a universal solution because developers did not know what users would try to do with the technology.
However, you know exactly what you want to achieve. At that point, the chat interface becomes the wrong tool.
Steven Pinker, a linguistics professor, noted that the first large-scale AI implementation was a gimmick. He argued there is tremendous promise if the technology focuses on tasks rather than conversation.
The canvas concept
The GitHub Copilot app addresses this limitation with a feature called a canvas.
A canvas is a full-stack application that runs inside the GitHub Copilot interface. It operates without a web browser. The agent communicates with the server part of this app, and the server sends data back. This creates a surface capable of performing any standard computer program task while maintaining a two-way connection with the agent.
The GitHub Copilot app recognises this structure. You do not need to explain the concept beyond asking for it.
Playing games
One simple example involves creating a Connect 4 game. The user interacts with the canvas while the agent controls the game logic.
The agent can execute code locally on your machine. It can also call third-party APIs.
For instance, a UI for Winget allows you to browse the registry for packages. You can manage local packages, including installation and removal.
There is no AI involved in this specific function. The point is that the agent builds a tool where future interactions are free. This avoids wasting tokens by treating the agent itself as the tool.
Instead of asking the agent to stage and commit code, you can use a canvas to build a tool that performs those actions.
Database and writing tools
You can build a canvas to interact with a SQLite database. You can write SQL queries yourself within this interface. The system provides IntelliSense support for the code.
Another example restores the Windows Live Writer interface. You can write blog posts in plain Markdown within this revived tool.
Automating workflows
The value of a custom interface becomes clear when automating development workflows.
A typical process involves research, prototyping, planning, implementation, iteration, and finalization.
Each step requires you to be at the keyboard. You must interact with prototypes and guide the process.
However, you do not need to be present for the entire duration. The agent can research and generate prototypes, then notify you when a review is ready. The goal is to remove yourself from the loop as much as possible.
This is difficult when the only interaction method is a chat box. A canvas allows you to automate the flow while keeping human gates where necessary.
One example shows an agent handling the research and prototyping phases. The agent waits for human approval before proceeding to implementation.
Being restricted to a chat interface makes it hard to solve actual problems. It is not obvious what to do when the only option is a text box.
Try using canvases for tasks like database management. These can be set up quickly. More complex workflows, such as the automation example, may take a day to design correctly.
You can achieve further results with AI when you think outside the chat box.
