Using 123API in AnythingLLM

Use 123API as the OpenAI-compatible provider in AnythingLLM for chat, knowledge base and retrieval workflows.

Prerequisites

1. Install AnythingLLM on your machine or server
2. Create a 123API API key from the dashboard 
3. Confirm your preferred chat model is available in GET /v1/models

Step 1: Open Settings

After launching AnythingLLM:

1. Open the application
2. Click Settings
3. Enter the LLM Preference section

Step 2: Configure Chat Model

Choose Generic OpenAI or the OpenAI-compatible provider option and fill in:

Step 3: Configure Embeddings

If you plan to use document retrieval or a private knowledge base, also configure an embedding model:

Step 4: Create a Workspace

1. Create a new workspace
2. Upload a small batch of test documents
3. Wait for chunking and vectorization to finish
4. Send a question based on those documents and verify retrieval quality

Recommended Validation

1. Save the provider configuration
2. Create a test workspace
3. Send a simple prompt and confirm the response returns successfully
4. If embeddings or RAG flows are enabled, validate those endpoints separately

Recommended Models

• gpt-4o-mini for low-cost testing and simple assistant flows
• gpt-4o for higher-quality daily conversations
• gpt-5 for harder reasoning or task planning
• claude-sonnet for long context and analysis-heavy workflows

FAQ