Lazy AI Providers in Angular: From Cloud Requests to Local Inference

An AI feature can start with a small HTTP request and end with a model running on the user’s GPU. Those two implementations have very different startup costs. Loading both when the application opens makes the lightweight path pay for a feature the user might never use.

The ng-switzerland demo explores this problem with Angular 22: one chat interface, two providers, and an explicit boundary between selecting a provider and loading it. OpenAI uses a separate NestJS API. Local AI uses WebLLM and a Llama 3.2 1B model inside a Web Worker.

One contract, two execution paths

The UI needs to submit a prompt and display a response. It does not need to know how an HTTP adapter works or how a local model initializes. Both providers implement the same TypeScript interface:

export interface AiProvider {
  chat(prompt: string, model?: string, onProgress?: (status: string) => void): Promise<string>;
}

export type AiProviderId = 'openai' | 'local';
export type AiProviderLoader = () => Promise<AiProvider>;

export const AI_PROVIDER_OPTIONS = [
  { id: 'openai', label: 'OpenAI' },
  { id: 'local', label: 'Local AI' },
] as const satisfies readonly { id: AiProviderId; label: string }[];

export const OPENAI_MODELS = [
  { id: 'gpt-4.1-mini', label: 'GPT-4.1 mini' },
  { id: 'gpt-5-mini', label: 'GPT-5 mini' },
] as const;

export const LOCAL_MODEL = 'Llama-3.2-1B-Instruct-q4f16_1-MLC';

The progress callback gives the local provider a way to report initialization without importing UI state. The interface is erased at runtime; an injection token supplies the actual provider loader.

Return a loader before loading the service

The key detail in Angular’s injectAsync API is its return type: a function that resolves a dependency asynchronously. The demo creates those functions in an injection context and chooses between them with a computed signal:

import { computed, inject, injectAsync, InjectionToken, type Signal } from '@angular/core';
import type { AiProvider, AiProviderId, AiProviderLoader } from './ai-provider';
import { AI_PROVIDER_SELECTION } from './ai-provider-selection';

export const AI_PROVIDER_LOADER = new InjectionToken<Signal<AiProviderLoader>>(
  'AI_PROVIDER_LOADER',
);

export function createAiProviderLoader(): Signal<AiProviderLoader> {
  const selection = inject(AI_PROVIDER_SELECTION);
  const loaders = {
    openai: injectAsync<AiProvider>(() =>
      import('./open-ai-provider').then((m) => m.OpenAiProvider),
    ),
    local: injectAsync<AiProvider>(() =>
      import('./local-ai-provider').then((m) => m.LocalAiProvider),
    ),
  } satisfies Record<AiProviderId, AiProviderLoader>;

  return computed(() => loaders[selection()]);
}

Changing selection changes which function the signal returns. Calling that function is the point where the selected dependency is requested. This is why choosing Local AI in the dropdown does not, by itself, initialize a model.

The concrete providers are auto-provided with @Service() in this Angular 22 project. Keep their imports dynamic and use type-only imports for the shared contract. Statically registering or importing a concrete provider elsewhere can undermine the intended loading boundary. See Angular’s lazy service loading guide for the framework requirements.

Capture selection before awaiting

A user can change the dropdown while a request is in flight. That should affect the next request, while the current one continues with the provider it started with. The facade captures the loader synchronously:

import { inject, Injectable } from '@angular/core';
import type { AiProvider } from './ai-provider';
import { AI_PROVIDER_LOADER } from './ai-provider-loader';

@Injectable()
export class Ai implements AiProvider {
  readonly #provider = inject(AI_PROVIDER_LOADER);

  async chat(
    prompt: string,
    model?: string,
    onProgress?: (status: string) => void,
  ): Promise<string> {
    // Capture the selection before awaiting: a pending request keeps its provider.
    const loadProvider = this.#provider();
    const provider = await loadProvider();
    return provider.chat(prompt, model, onProgress);
  }
}

This small ordering choice defines predictable behavior: select, capture, await, execute. The facade delegates model initialization and generation to the chosen provider.

Choose scope deliberately

provideAi() registers the facade, selection signal, and loader together. The chat component receives its own selection chain through provideAiChat(). Two chat instances can therefore choose different providers.

The concrete adapters are root-scoped. The OpenAI adapter is stateless; the local provider owns a reusable runtime. Sharing the local runtime avoids creating a separate GPU engine for every chat, but it also means concurrent generation needs a policy. This demo rejects another local request while that runtime is busy.

Loading JavaScript is not loading the model

Local AI has several stages: loading the provider chunk, importing WebLLM, starting a worker, downloading model assets, and initializing the engine. The provider creates its runtime on the first chat() call and reuses it for later calls.

The first initialization can download hundreds of megabytes. WebGPU support and sufficient GPU memory are required. Progress is surfaced to the composer; initialization and generation have a five-minute timeout. Failures reset the runtime so the next request can retry, and destruction terminates the worker.

The demo uses a 4096-token context window and caps generated output at 512 tokens. Each prompt is independent: the transcript displayed in the UI is not sent as conversation context. A chat-shaped interface does not automatically imply conversational memory.

Keep the PWA from undoing the loading boundary

A dynamic import can be deferred while a service worker still eagerly downloads its chunk. In this project, the main application assets use prefetch installation, while the remaining JavaScript assets use lazy installation and updates.

That keeps the local engine off the initial download path. It does not guarantee offline inference. WebLLM manages its model cache separately, and offline use depends on the required assets remaining available. Browser storage eviction can require a fresh download.

The frontend prerenders its home page and hydrates in the browser. GPU access happens only when local inference is requested. Real OpenAI calls need connectivity and a configured backend; the frontend production configuration currently uses mock mode until a hosted API URL is supplied. API credentials stay on the server.

Verify the boundary in the network panel

1. Start a fresh browser session and clear the network log.
2. Submit an OpenAI prompt and check that no local runtime or model assets load.
3. Select Local AI and confirm model initialization still has not started.
4. Submit a local prompt and observe the provider, worker, runtime, and model requests.
5. Submit another local prompt and check runtime reuse.

The project’s automated tests use doubles to cover provider selection, independent chat scopes, HTTP errors, and the local runtime lifecycle. They do not establish real GPU inference on every supported device. Verify that separately on the browsers and hardware you intend to use.

The design lesson

The useful boundary is the moment an expensive capability becomes necessary. In this demo, selection is cheap, service resolution is deferred, and model initialization belongs to the runtime that owns it. That separation lets one interface support both a small cloud adapter and a substantial local engine without forcing every user to load both.