Choosing a model
Start from the job and pick a model that is good enough without overpaying.
Start from the job and pick a model that is good enough without overpaying.
The first image looks right, but a new scene seems to introduce a different actor. Each result may look attractive on its own while the collection fails to depict one recognizable character. For virtual personas, model imagery, and storyboards, continuity across the set matters.
Here, character consistency means visual identity across images, rather than the speaking style of a chatbot. It requires managing reference material, generation conditions, and acceptance criteria together. A request to keep the same person is only one part of that process.
Create a short character specification that distinguishes identity from the requirements of the current image.
| Type | Examples | How to manage it |
|---|---|---|
| Identity baseline | Facial outline, relative facial features, distinctive identifiers | Use the approved character material as the baseline |
| Project continuity | Hair, clothing, accessories, visual style | Decide which details must persist and which may change |
| Shot variables | Setting, action, composition, lighting | Describe the changes required for this image |
The same person does not necessarily mean every detail remains unchanged. An outfit project may require different clothing, while a continuous story may require one outfit throughout. Establish that boundary before reviewing results.
Choose one clear baseline without obvious generation errors. Add other views or body-proportion information when the planned shots require them. More references are not automatically better.
Check whether:
Keep backgrounds simple enough to reduce distractions. A pure white background is not a universal model requirement. Follow the target model's rules for reference count, dimensions, and input format.
Retain an approved baseline. Generated images can become candidate references, but compare them with that baseline first. Otherwise, an unnoticed change can become the starting point for further edits.
First check whether a model accepts reference images, supports editing, and provides the multi-image or local-editing controls you need. Text-only generation and reference-based editing are different operations.
The FLUX.1 Kontext introduction from Black Forest Labs describes character preservation and targeted editing. Google's image prompting guide also demonstrates maintaining a character in different contexts. These are capability references, not a ranking for your particular inputs.
Compare candidate models on the same tasks: identity retention, instruction following, repairs required, and cost per deliverable image.
For compatible diffusion workflows, image-conditioning components are another option; see the Diffusers IP-Adapter documentation. These methods need a supporting pipeline. An arbitrary parameter added to a commercial API does not enable them.
A reusable structure identifies the supplied character, then describes the shot and the conditions that should persist.
The following is an example written for this article. It is not a measured result or a model-specific parameter format:
Use the same character shown in the attached reference images.
Preserve the character's facial identity.
This image: the character sits beside a bookshop window,
reading an open book.
Framing: a waist-up medium shot at approximately eye level.
Allowed changes: the setting, seated pose, and a light gray cardigan.
Preserve: the character's identity and established hairstyle.
Style: a natural documentary photograph with soft window light
and restrained retouching.
When the reference already establishes the face, a new detailed facial specification in every request may be unnecessary. Resolve contradictions between text and images. If you deliberately change hair, apparent age, or another appearance condition, state that intention and update the acceptance criteria.
The photographic style should match the job. A casual portrait may need unpolished framing, while a product shoot may need controlled lighting. Neither a phone-photo look nor a studio look is a universal consistency rule.
Aspect ratio also changes framing, subject size, and cropping. Start comparisons at one ratio, then test other formats after the baseline works.
The following is a proposed checking procedure, not a validated industry benchmark.
| Shot | Main question |
|---|---|
| Familiar angle with a simple background | Does the baseline identity survive? |
| Changed angle or action | Does the character drift in profile or expression? |
| Wider framing or a different setting | Does the character still meet the brief when smaller or surrounded by more detail? |
Change one condition first, then combine changes. Changing clothing, apparent age, style, angle, and lighting together makes a failed result harder to diagnose.
To compare two models, you could generate three shots twice with each model, for twelve requests in total. This is a manageable starting budget, not enough evidence to claim that one model is universally better.
Save the model and version, reference identifiers, complete prompt, aspect ratio, quality setting, and actual charges. Record a seed when the endpoint exposes one, without treating it as an identity guarantee across prompts or models.
Expand the run after the sample passes. Repeat the important shots when changing the baseline references or model version.
Attractive lighting can hide a changed face. Use a fixed review order:
Assign a clear next action:
Attach a short reason, such as a changed profile or an otherwise acceptable image with a hand defect. Do not promote an attractive but incorrect identity into the reference library just because the image looks polished.
When identity and composition pass, consider supported local editing for a hand, text element, or background detail. Preserve the original as a rollback point and inspect the whole result after editing.
If identity has already changed substantially, return to the baseline and original request conditions before deciding how to regenerate. Repeated edits to an incorrect starting point can make diagnosis harder. BFL also documents the possibility of visual artifacts after excessive iterative editing.
Simple folders and a record sheet are enough to start managing a batch. Preserve the source request, review state, and reason for each revision. Move approved results into a delivery folder without depending on a particular application's gallery or queue controls.
Track two quantities:
Deliverable share = accepted images / images reviewed in this batch
Cost per deliverable = actual total batch spending / final deliverable images
Include actual generation, repair, upscaling, and other charges. If a hypothetical batch costs CNY 12 and produces six accepted images, the cost per deliverable is CNY 2. These numbers illustrate the calculation, not model pricing or measured performance. With no accepted images, do not report a zero cost per deliverable.
Results depend on the model, character, and review standard. Another user's acceptance rate does not establish your budget. Rates produced by different standards are not directly comparable.
Yes, particularly when the angle and framing stay close to the reference. Add views as the planned shots and model behavior require. Multiple images do not automatically guarantee better identity retention.
Not necessarily when the references already establish them. Repeated descriptions may be unnecessary, while contradictions make the target less clear. Use text to identify the source character, current shot, and intended changes.
There is no guarantee. Quality settings, output size, and identity retention are different dimensions. Check the references, editing mode, and instructions too, and use quality controls only where the model actually provides them.
There is no general guarantee. Seed behavior depends on the model and implementation. Changes to the prompt, model, or other settings still require review.
No. It makes generation conditions easier to control and helps identify drift and select deliverables. Verify the outcome using your own references, tasks, and acceptance criteria.
Before a larger run, check the target model's image input, editing, dimensions, and pricing in the API platform. Start with a sample before increasing volume. For programmatic batches, see the API 429 rate-limit and retry troubleshooting guide.
This article reorganizes the problem around references, prompting, review, and cost control. Its sample prompt and checking procedure were written for this guide and are not reported experiments from the source. The workflow was informed by this guide to consistency across a character batch; capability references are linked in the relevant sections above.
Your API works during testing, but production starts returning 429 Too Many Requests. The cause is not always a sudden increase in users. Long prompts can exhaust token capacity, several services may share one limit, and automatic retries can turn a brief traffic spike into repeated failures.
Start with the error body: identify what is being limited before reducing request traffic, cutting token usage, or addressing a quota problem. This guide explains RPM, TPM, and concurrency, walks through common causes, and provides a Python retry example with exponential backoff, jitter, and Retry-After handling.
Retry-After is present, wait at least that long. Otherwise use bounded exponential backoff with jitter. Persistent failures require changes to release rates, token usage, or capacity.See MDN's description of HTTP 429 for the status and possible rate-limit scopes. Use the current service's error body to determine the specific cause.
Request count alone does not describe the work an LLM must perform.
A short classification call might contain 200 tokens and return one label. A repository analysis might include thousands of input tokens and produce a long response. Both count as one request, but their token usage, execution time, and compute requirements differ.
Different limits address different traffic patterns:
Common metrics include RPM, TPM, and hourly or daily request and token limits. Image services may also use IPM and IPD for images per minute and per day. A request can be restricted when any applicable limit is reached. Not every provider uses every metric.
RPM and TPM measure different dimensions of usage.
| Metric | Meaning | Common Bottleneck |
|---|---|---|
| RPM | Requests per minute | Frequent small requests |
| TPM | Tokens per minute | Long prompts, large contexts, or long outputs |
| RPH | Requests per hour | Hourly usage limits |
| RPD | Requests per day | Daily request allowances |
| TPD | Tokens per day | Daily token consumption |
RPM limits the requests allowed each minute within a particular scope. That scope may be an account, project, model, or another resource, depending on the service.
An application can reach RPM even when every prompt is short. Autocomplete, chat applications, agent loops, and workflows that split one user action into several model calls are common examples.
An agent might make separate calls for planning, tool selection, validation, and the final response. The user submits one task, while the application makes four or more requests.
TPM limits the tokens counted toward rate limiting each minute. System prompts, conversation history, retrieved documents, code, reasoning content, and long outputs can increase consumption.
Record observed prompt_tokens, completion_tokens, and total_tokens, or the equivalent fields in your protocol. Rate-limit accounting is not necessarily identical to final billing usage. Some services consider estimated input or the requested output limit. Check the applicable documentation.
Consider two applications making ten requests per minute:
Both use 10 RPM, but Application B is more likely to reach TPM first.
Verify the scope before planning capacity. Some services share capacity at the account or project level, with additional restrictions by model, model group, key, or other resource.
Creating more API keys does not necessarily create more capacity. If the keys share an account, project, or upstream allowance, total capacity may stay unchanged.
Do not assume different models have independent limits, or that a plan upgrade increases every metric. Check the target model, account tier, and effective limits before deployment.
A 429 commonly indicates an applicable usage restriction, but the status alone does not identify which restriction was reached. Read the error code, message, and response headers.
A burst of small calls may exceed RPM. Agent workflows can generate many more API calls than the number of tasks visible in the interface.
A few long requests may exceed TPM. Repeated system prompts, full conversation histories, large retrieval results, long code files, and unnecessary output allowances are frequent causes.
Applications, backend services, workers, and keys may share account, project, model, or upstream capacity. Each service can appear lightly loaded while their combined usage reaches the limit.
Low hourly or daily averages do not rule out a busy minute. Scheduled jobs starting together or a queue releasing many tasks at once can create short-term peaks.
Check queue depth, request release rates, and minute-level usage alongside daily totals.
Immediate retries send additional traffic into a system that is already restricted. Workers that fail together can retry together, creating a retry storm and prolonging the problem.
Some models or plans have hourly or daily request and token limits. Staying below RPM and TPM does not guarantee that other allowances remain available.
For a dedicated deployment, verify the endpoint, deployment name, model ID, and key permissions. A request accidentally sent to a shared endpoint may encounter a different set of restrictions.
Whether dedicated capacity removes particular shared limits, and which gateway or instance limits remain, depends on the deployment. Dedicated does not mean unlimited throughput.
If the error identifies a balance, billing, permission, or exhausted long-term quota issue, address that condition before retrying. When details are insufficient, provide the service operator with the request ID, time, and redacted error response.
Not every temporary API failure is a rate-limit error. Identify the status first, then inspect the error details.
| Status | Common Meaning | First Action |
|---|---|---|
| 429 | Request, token, or other usage restriction; some services also report quota issues this way | Identify the restriction; throttle and back off for temporary limits, or resolve the quota issue |
| 503 | Temporary service unavailability, possibly due to load or maintenance | Check service status; retry suitable tasks within limits or use a validated fallback |
| 504 | A gateway did not receive an upstream response in time | Check task status, duration, and timeouts; avoid resubmitting work already in progress |
| 400 | Invalid request format or parameters | Correct the request using the error body; invalid model IDs may use other status codes |
| 401 | Missing or invalid authentication | Check the key, authentication header, and destination service |
| 403 | Insufficient permission | Check account, project, and model access |
| 404 | Missing endpoint, model, or resource | Check the base URL, path construction, API protocol, and resource ID |
Repeated 504 errors on long non-streaming calls may not improve if the same call is simply submitted again. Where supported, evaluate streaming, shorter outputs, or changes to client and gateway timeouts.
Streaming can keep data moving, but does not increase RPM or TPM and cannot prevent every upstream execution timeout. For asynchronous jobs, check the existing task status before submitting another job.
Retry only suitable temporary failures, with limits on attempts, waiting, and overall business duration. Establish whether repeating the operation is acceptable before selecting a retry policy.
Increase the delay after consecutive failures, for example approximately one, two, four, and eight seconds. Bound both the delay and total attempts. See the AWS explanation of backoff and jitter for the underlying approach.
Add a random offset to the delay so clients do not all retry at the same moment. Otherwise requests that fail together may arrive together again.
A queue absorbs bursts and releases work at a sustainable rate. Interactive requests can receive priority while background jobs wait.
Token buckets and leaky buckets control when new requests are released. A concurrency cap controls how many calls are active. These mechanisms solve different problems and often work together.
For some applications, three to five total attempts can be a starting point, adjusted for latency and business cost. Five attempts includes the initial request; it does not mean five additional retries.
After the last failure, return a clear error, defer the job, or use an approved fallback. Interactive work also needs an overall deadline so it does not continue long after the user has left.
Agents may call payment systems, databases, file operations, or other tools. Retrying a model call should not automatically repeat a business operation that already succeeded.
Use idempotency keys, operation IDs, task state, and deduplication to protect downstream actions. A network timeout does not prove that the server did nothing; another generation request may incur another charge.
Retries in both the SDK and application can multiply attempts. Disable SDK retries when applying an application policy, or account for both layers in the budget. The official OpenAI rate-limit guide describes these retry boundaries.
Retry-After can contain a delay in seconds or an HTTP date. A valid server hint specifies the minimum wait. If that wait exceeds the task's budget, stop and defer the task rather than retrying earlier. See MDN's Retry-After reference for the formats.
The following Python example targets an OpenAI-compatible Chat Completions endpoint. It disables SDK retries, bounds total attempts and cumulative retry sleep, and stops immediately for recognized quota or billing error codes.
Install openai and configure API_KEY, API_BASE_URL, and MODEL_ID. Use the full base URL supplied by the service documentation, and adapt parameters such as max_tokens to the target model. The example requires Python 3.10 or later.
import math
import os
import random
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from openai import APIConnectionError, APIStatusError, OpenAI
client = OpenAI(
api_key=os.environ["API_KEY"],
base_url=os.environ["API_BASE_URL"],
max_retries=0,
timeout=30.0,
)
# Extend this set with the documented error codes of your API provider.
NON_RETRYABLE_CODES = {
"insufficient_quota",
"billing_hard_limit_reached",
"billing_not_active",
}
def parse_retry_after(value: str | None) -> float | None:
if not value:
return None
try:
seconds = float(value)
return max(0.0, seconds) if math.isfinite(seconds) else None
except ValueError:
pass
try:
retry_time = parsedate_to_datetime(value)
if retry_time.tzinfo is None:
retry_time = retry_time.replace(tzinfo=timezone.utc)
seconds = (retry_time - datetime.now(timezone.utc)).total_seconds()
return max(0.0, seconds)
except (TypeError, ValueError, OverflowError):
return None
def create_chat_completion(
messages: list[dict[str, str]],
max_attempts: int = 5,
max_retry_wait: float = 60.0,
):
if max_attempts < 1 or max_retry_wait < 0:
raise ValueError("Invalid retry limits")
waited = 0.0
for attempt in range(max_attempts):
try:
return client.chat.completions.create(
model=os.environ["MODEL_ID"],
messages=messages,
max_tokens=800,
)
except APIStatusError as error:
if (
error.status_code not in {429, 503, 504}
or error.code in NON_RETRYABLE_CODES
or attempt == max_attempts - 1
):
raise
backoff = min(30.0, 2.0 ** attempt)
server_delay = parse_retry_after(
error.response.headers.get("retry-after")
)
delay = server_delay if server_delay is not None else backoff
delay += random.uniform(0.0, min(1.0, delay * 0.25))
if waited + delay > max_retry_wait:
raise
except APIConnectionError:
if attempt == max_attempts - 1:
raise
delay = min(30.0, 2.0 ** attempt) + random.uniform(0.0, 1.0)
if waited + delay > max_retry_wait:
raise
time.sleep(delay)
waited += delay
raise RuntimeError("The request failed after all retry attempts.")
max_attempts=5 permits at most five attempts. max_retry_wait=60 limits cumulative sleep only, excluding network time. Likewise, timeout=30.0 is not a 30-second deadline for the entire business operation. Implement overall cancellation and deadlines in the calling application.
Provider error codes vary: extend NON_RETRYABLE_CODES using the actual documentation. The example is not a replay policy for partially consumed streams and does not determine whether an asynchronous task or external operation already succeeded.
RPM and TPM measure throughput. Concurrency measures active requests. They cannot be substituted directly.
Assuming the token limit uses total-token accounting, observed data provides a rough starting point:
RPM-based request rate = RPM / 60
TPM-based request rate = TPM / observed total tokens per request / 60
Planned release rate = min(RPM-based rate, TPM-based rate) * safety factor
Concurrency planning estimate = planned release rate * request latency
Suppose the limits and observations are:
The RPM-based rate is:
1,200 / 60 = 20 requests per second
The TPM-based rate is:
60,000 / 1,200 / 60 ≈ 0.83 requests per second
TPM is the tighter constraint. Applying the safety factor:
0.83 * 0.8 ≈ 0.67 requests per second
Using P95 latency for a concurrency planning estimate:
0.67 * 4 ≈ 2.68 active requests
You could begin observing behavior with a cap of two or three concurrent calls while limiting release to approximately 0.67 requests per second. These are illustrative numbers, not any platform's actual limits.
This is only a capacity-planning starting point. Average token usage misses long-request tails, and multiplying by P95 latency is a planning heuristic rather than a strict concurrency guarantee. Request lengths, latency, server windows, and other restrictions affect the outcome.
A concurrency cap alone does not enforce RPM or TPM. Fast requests can finish and immediately be replaced, exceeding the minute-level allowance. Add request throttling and token controls that match the service's accounting.
Include rate-limit handling in normal traffic design instead of adding a retry loop only after production failures.
One interface action may involve planning, retrieval, tool use, and validation across multiple model calls. Plan capacity using actual call counts.
Track median, P95, and maximum input, output, and total tokens. A small number of very long requests can cause problems even when average usage looks normal.
Use per-model queues and concurrency controls when models have independent allowances. Add a shared cap when models share account or upstream capacity. A single global cap and fully independent queues can both be wrong if they do not match the real rules.
Remove repeated context, limit retrieval results, summarize older conversation history, and choose realistic output allowances. These changes can reduce token pressure and inference costs together.
Choose models for classification, extraction, reasoning, coding, and multimodal work according to quality, latency, cost, and available capacity.
Validate fallback quality and API compatibility. Switching models should not be treated as a way to circumvent service restrictions, and another model does not necessarily have independent capacity.
Evaluations, document processing, classification, and large extraction jobs do not always require immediate results. Where a batch API is available, check supported models, endpoints, completion windows, prices, and limits.
Without a batch API, your own queue can process jobs at the allowed rate. Sending ordinary requests in a loop does not automatically provide a batch discount or additional capacity.
Prompt optimization, queues, and bounded retries can manage short bursts, but cannot solve a persistent mismatch between demand and capacity.
When normal traffic stays near its limits, follow the service's process for requesting higher RPM, TPM, or other capacity. Provide expected traffic, input and output distributions, peaks, workload types, and latency requirements. Submitting a request does not guarantee approval.
For sustained high throughput or more predictable capacity, evaluate available dedicated deployments. Compare throughput, latency, queueing, costs, and remaining gateway or instance restrictions. Do not assume dedicated capacity is unlimited.
TPM means tokens per minute. Long prompts, history, retrieved documents, reasoning, and long outputs can cause an application to reach its token allowance before reaching RPM.
A few long calls can consume substantial token capacity. Bursts, shared allowances, nested retries, and hourly or daily limits can also trigger 429. Read the error body rather than judging usage from user counts alone.
No. Suitable temporary rate limits, selected 503 or 504 responses, and connection errors may justify bounded backoff. Correct invalid parameters, authentication, permissions, paths, billing, and exhausted long-term quotas first. After a timeout, determine whether the operation already ran.
Not necessarily. They are separate constraints and can vary by model, account, project, and deployment. Confirm each when requesting capacity.
Control request release, concurrency, and token usage. Use queues and bounded retries with jitter, shorten unnecessary context, and move suitable offline work away from real-time calls. Request more capacity or evaluate another deployment when sustained demand exceeds the available allowance.
When you are ready to continue integrating, open the API platform and verify the base URL, model ID, and credentials against the current integration documentation. Validate the configuration with one minimal request. Include a timestamp, request ID, and redacted error body when asking for troubleshooting help.
Related reading: Choosing a model.
DeepSeek officially released DeepSeek V4.1 Flash on September 10. According to the company, the new version beats V4 Pro on performance, speed, cost and total task time, and after September 14 all V4 Pro API requests will be routed to V4.1 Flash.
This is not a routine point release. V4.1 Flash moves to a new Causal-Encoder-Decoder architecture with 552B total parameters, activating only 8B on the input side and 16B on the output side. It also adds native visual understanding, shrinks the KV Cache substantially once again and cuts API prices further. DeepSeek's intent is clear: have a Flash model take on more and more of the work that used to belong to Pro, while pushing inference costs down even further.
V4.1 Flash is the smallest model in DeepSeek's new architecture family. It has 552B total parameters in a mixture-of-experts (MoE) design, which makes it well suited to agent workloads. A coding agent typically reads code repositories, tool documentation and conversation history over and over, then outputs only a small amount of code or a few tool commands. Using different compute scales for input and output cuts the cost of the reading stage.
The official benchmarks show V4.1 Flash performing strongly on GPQA Diamond, Codeforces, MathArena Apex, Terminal-Bench and CyberGym, beating V4 Pro on some of them. On Terminal-Bench 3.0/4.0 and ProgramBench, however, it still trails the top closed models. Overall, the upgrade is still about balancing capability, speed and cost.
DeepSeek previously offered vision mainly through experimental releases such as V4-Flash-Vision-Exp. This time V4.1 Flash folds visual understanding directly into the main model, so images, screenshots, charts and PDFs can all go through the same model pipeline.
That matters especially for agents. Real-world tasks are rarely text-only: web pages contain screenshots, product docs contain flowcharts, coding tasks involve error screens, and office files come with spreadsheets and PDFs. A model that can handle all of this within a single task context is far easier to work with.
The new generation shrinks the KV Cache dramatically: compared with the previous generation, HBM requirements drop to 1/4 and SSD requirements to 1/8; compared with the original model, the KV Cache is now 437 times smaller. If cache footprint and cache-hit prices keep falling, the cost of an agent running dozens or hundreds of consecutive rounds falls with them. V4.1 Flash optimizes caching, architecture and pricing together, with the goal of making the model better suited to sustained work.
The V4.1 Flash API keeps peak and off-peak pricing, with off-peak prices at half the peak rate; the new prices took effect at 12:00 on September 10. DeepSeek also announced that V4 Pro would be phased out after 12:00 on September 14, with requests that used to go to V4 Pro routed directly to V4.1 Flash. The move makes the new model's role plain: Flash is stepping into the main-model slot.
On the current Artificial Analysis leaderboard, V4.1 Flash ranks 37th overall and 6th among Flash models.
Beyond specs, benchmarks and pricing, differences in coding, agent, multimodal and long-context work only show up when the model is actually put to work. The following side-by-side tests check whether V4.1 Flash lives up to the promise of "Pro-level capability at a Flash-level price" on real tasks.
deepseek-v4.1-flash, with a 1M-token context window (1,000,000 tokens).
The evaluation focuses on logic, math, coding, multimodal and human-intuition questions. It is not an authoritative test of cutting-edge specialist domains; the goal is to track how models are evolving and offer a reference for choosing between them.
Coding cases are rated on the following scale:
DeepSeek-V4.1-Flash posted a weighted total of 37.84, ranking 16th on the scoreboard. Its category scores were 9.60 for logic and math, 9.46 for human intuition, 9.78 for coding simulation and 9.00 for multimodal.
Prompt 1: Draw an animated SVG of a carousel in motion.
Prompt 2: Draw an animated SVG of a young skateboarder popping an Ollie in midair.
Verdict: V4.1 Flash gets the compositional complexity and the atmosphere of the scenes about right, with noticeable flaws only in the animation. The skateboarder's leg joints change through the takeoff, the airborne phase and the landing, for example, but the hand movements are implausible.
Given the URL of the Mono e-commerce template on v0.app, the model was asked to recreate the page pixel for pixel.
Verdict: fidelity is high overall. The page's basic structure is reproduced in full and the animated effects largely match the original; differences remain only in the details below.
Font differences:
Layout differences:
Prompt:
Create a single-page HTML 3D animation using Three.js.
Build a clean product-style scene with a transparent rectangular box containing about 30 colorful balls.
Animation:
1. The box slowly tilts to one side, causing the balls to fall out naturally.
2. The balls roll, bounce, and collide with each other and the ground.
3. After a short pause, the balls gradually roll back toward the box.
4. The balls enter the box and settle back inside.
5. The animation then loops.
Requirements:
- Use Three.js and JavaScript.
- Simulate basic gravity, velocity, collision, bouncing, and friction.
- Avoid balls passing through each other, the box, or the ground.
- The balls should have natural physical movement rather than simply following predefined paths.
- Add soft lighting, shadows, and a clean light-colored background.
- Add Play / Replay controls.
- Allow mouse drag to rotate the camera and scroll to zoom.
- Make the layout responsive.
Focus on realistic physics and smooth transitions. Avoid clipping, floating, teleportation, or obviously scripted-looking movement.DeepSeek-V4-Pro's output for the same prompt served as the comparison.
Verdict: the ball-pouring simulator from DeepSeek-V4.1-Flash is clearly more complete than V4 Pro's in modeling quality, proportions and the control panel. It also handles the most important part, the motion, very well: the balls show no obvious violations of physical logic.
Prompt:
Create a polished, cinematic 3D railway simulator entirely in Three.js. Build a continuous countryside railway where a passenger train travels from a small station through a village, dense forest, mountain valley, dramatic bridge, and dark tunnel before emerging beside a beautiful lake.
Procedurally generate the train, tracks, terrain, trees, buildings, bridges, tunnels, signals, and other environmental details without Blender or external 3D assets. Give the train realistic motion with rotating wheels, subtle suspension, headlights, and smoke, while adding moving cars, birds, animated water, and swaying vegetation to make the world feel alive.
Use cinematic camera movements that follow the train, reveal wide landscapes, and showcase key moments along the journey. Make the environment gradually shift from warm countryside to mysterious forest, dramatic mountains, and a bright lakeside finale. Prioritize visual richness, smooth animation, atmospheric lighting, and a strong sense of scale while keeping everything performant in the browser.Strengths:
Weaknesses:
Although V4.1 carries the Flash suffix, it shows capabilities close to, and in places beyond, V4 Pro. Across the coding, 3D scene-building, physics simulation and multimodal tasks in this round, it delivered a high level of completeness and stability, and in some cases it actually outshone V4 Pro.
The most visible gains are in coding. The animated SVGs, the website recreation, the Three.js railway simulator and the ball physics simulation all show that V4.1 Flash has become better at breaking down and implementing complex requirements. The 3D ball-pouring case is the clearest example: the model had to handle scene modeling, gravity, collisions, bouncing, friction, box tilting and interactive controls all at once, and the result is fairly complete in modeling quality, proportions, control panel and motion, with no obvious violations of physical logic. Compared with V4 Pro, the improvement on this kind of composite coding task is especially apparent. In the complex Three.js case, the model was likewise able to build a large-scale scene, handle lighting, animation and environmental detail, and keep outputting, reasoning and repeatedly verifying its deliverable over a long-horizon task of 4,000 lines of code in a single file.
Native multimodality further improves the practical experience. Images, screenshots, charts and PDFs can go straight into the same workflow, which is more convenient for agents that need visual information to complete a task, and the output keeps its visual taste and overall style fairly well. Combined with lower API prices and a smaller KV Cache, V4.1 Flash also has an edge in long-context, multi-turn and sustained-execution scenarios.
V4.1 Flash beats V4 Pro on some specific tasks, but its ceiling is still some distance from that of true mainstream flagship models. It has been heavily optimized for complex coding, 3D scenes and agent tasks, yet stability still wavers once the difficulty goes up further, as with the derailed carriages, floating plants and widespread clipping in the railway simulator. The ball simulation is fairly complete overall, but that mostly shows it can now handle complex tasks; it still has a way to go before matching the sustained high quality, distinctive taste and low error rate of flagship models.
In terms of real-world cost, V4.1 Flash does not have an overwhelming price-performance edge either. API prices are certainly low, and cached input is especially attractive, but when capability, stability and price are weighed together, peers such as GLM 5.3 Flash are more competitive. Cost control across different types of tasks also cannot be judged on API unit prices alone.
Source: 302.AI
On September 9, 2026, just five days after releasing GPT-6 Astra, OpenAI launched two image models under ChatGPT Images 2.5.
OpenAI opened its announcement with a single figure: every week, people generate more than 3 billion images in ChatGPT and the GPT-Image API. That works out to roughly 5,000 images per second.
If AI image generation were still a niche toy, the focus of each iteration would naturally be the quality ceiling, and another 10% jump in quality would keep the AI crowd buzzing for days. But when 3 billion images a week is routine, the tool has become a productivity staple for many people. It is infrastructure, and one of the biggest pain points of infrastructure is throughput.
That is why the most important line in this update is this one: generation latency is down by up to 50%, and API pricing is unchanged.
This upgrade includes a structural change, so it is worth sorting out the versions first.
Previously, GPT-Image-2 was a single model that handled everything, whatever the complexity of the task. Now Images 2.5 is one model in ChatGPT but is split into two in the API:
Both models carry exactly the same token price ($30 per million image output tokens). Third-party estimates put the actual per-image cost of 2.5 at about a quarter of the previous generation's at the same high-quality setting: with better efficiency, each image burns fewer tokens.
In one sentence: instead of forcing users to choose between "fast" and "good," OpenAI has put each answer behind its own endpoint.
OpenAI's update covers four areas: reference-image fidelity, precise editing, multi-turn consistency, and intelligence and style. The first two are the easiest to be impressed by, but the real turning point is the third.
Reference-image fidelity. Give it a photo of a real person, and the generated image still shows the same person: facial features, expression and recognizability survive a change of style. One widely shared third-party test turned a rough cat doodle drawn with a mouse into a realistic photo, then used a single follow-up instruction to change only the eye color while everything else stayed exactly the same.
Precise editing. "Change only what you specify" has gone from a gamble to a promise. Swap one product in a poster or change one outfit in a party photo, and the composition, lighting and surrounding details stay intact. For e-commerce and marketing teams, editing an image no longer means starting over.
Multi-turn editing consistency. Anyone who has used AI image tools knows the drill: round one, you are happy with the result; round two, you change the background and the face goes slightly off; round three, you try to fix the face and the outfit's style disappears. Every edit is a fresh roll of the dice, and after ten rounds the image is unrecognizable.
Images 2.5 remembers earlier changes across a multi-turn conversation, so each edit builds on what is already there instead of tearing it down and starting again.
This is the turning point because real creative work is never one-and-done. It is a loop of "generate, dislike, revise, revise again." Multi-turn editing that holds together means AI image generation can support a workflow for the first time, not just inspiration. Inspiration is a one-off; workflow is productivity.
On both the Artificial Analysis text-to-image and image editing leaderboards, the two GPT-Image-2.5 models take the top two spots.
The tests pit GPT-Image-2.5 Sunburst against its predecessor, GPT-Image-2, which had long held the state-of-the-art (SOTA) position, to see whether the improvements are obvious at a glance.
Prompt:
Fashion close-up editorial:
An Asian woman in her early 20s, wet-look styling with messy long black hair clinging to her face, sunburn makeup (heavy red blush across the bridge of the nose and the cheeks), glossy lips, realistic skin texture with freckles and pores.
Body turned sideways, looking back directly at the camera, calm and slightly hazy gaze, lips slightly parted. Bare shoulders, tightly hugging a huge cloud-like mass of white ruffled fabric, chin resting lightly on it.
Surrounded by giant surreal flowers, with the foreground and background framing her head together. Blue sky and dark green foliage in the background.
Hard midday light, strong highlights on skin and petals, sharp shadows, a midsummer-afternoon mood.
Highly saturated fashion surrealism, strong pink, orange, blue and green contrast, 35mm film grain, realistic skin detail with no smoothing.
| Model | GPT-Image-2.5 Sunburst | GPT-Image-2 |
|---|---|---|
| Score | 5/5 | 4.5/5 |
Verdict: both follow the prompt accurately, with realistic, attractive images; which one you prefer comes down to taste. From a photographic point of view, 2.5 renders the flowers in richer detail, including stamens and foreground elements, giving fuller depth; the light falling on the model's back also adds a stronger sense of dimension.
Prompt:
A colorful, abstract floral photograph with a strong motion-blur effect. The frame is filled with patches of pink, yellow, blue and green formed by flowers, stems and leaves blending and bleeding into one another. In the center of the frame, the lowercase text "more art merch coming soon" is written in a white serif font. The overall mood is light and whimsical. There is a small round white logo in the lower-right corner.
| Model | GPT-Image-2.5 Sunburst | GPT-Image-2 |
|---|---|---|
| Score | 5/5 | 3/5 |
Verdict: 2.5 wins on two counts. First, the defocus and slow-shutter effect stay within a reasonable range, so it is easy to tell the subject is flowers (GPT-Image-2's image is clearly too abstract). Second, the text and logo are restrained in size and do not steal the show, and the layout is sensible.
Prompt:
A surreal photograph of a man in a modern office, shot in an inverted composition, as if gravity has disappeared.
He wears a dark blue formal suit. His head is close to the floor and his legs extend upward toward the ceiling. His hands are raised near his torso, palms up, his body showing a natural tension, as if hovering in midair.
The office environment is completely normal and realistic: nothing in the room has been altered, and it looks like a typical everyday office. There is a wooden desk, and the walls are painted a very light, pale green. Only the man and the objects around him float in midair, creating a strong sense of weightlessness and controlled chaos.
A white desk phone with a coiled cord is caught in a dynamic moment of splitting apart: the handset drifts in one direction while the base floats on its own. A coffee cup hovers nearby. Other office supplies are floating too: a bulky old-fashioned computer monitor, a matching vintage keyboard, a mouse and an office chair, drifting in different poses, slightly rotating and tilting. A potted red geranium also hangs in the air, adding a vivid touch of color contrast.
The window is open, and bright midday sunlight pours into the room. Outside, the view of Azadi Square in Tehran is clearly visible. A pigeon flies into the room and also hovers in midair, wings slightly spread, as if frozen mid-motion. The lighting is bright, high-key and clean, with a minimalist aesthetic. The color palette is soft, slightly desaturated and cinematic. The overall mood emphasizes surrealism, weightlessness and a frozen moment.
| Model | GPT-Image-2.5 Sunburst | GPT-Image-2 |
|---|---|---|
| Score | 4.5/5 | 4/5 |
Verdict: it is obvious at a glance that the two images come from the same model family, with highly similar color tones, spatial layout and composition. 2.5's flaws: the office chair in the upper left is twisted and distorted, and the phone cord is not continuous, looking disconnected.
Prompt:
An expressive oil portrait: a young woman looking directly at the viewer over her bare shoulder, in a close-up composition. Fair skin with distinct texture, visible freckles, a natural flush on the cheeks and the tip of the nose, gray-green eyes, thick natural eyebrows. Her expression: eyes lowered, dignified and reserved, lashes casting soft shadows. Her hair is slicked back, glossy and smooth, revealing her whole face. She wears a charcoal-gray blazer, with her collarbones exposed.
Background: a sage/olive green backdrop with visible painterly brushstroke texture and no scenery. Soft, diffused natural light comes in from the front left, creating gentle modeling transitions on the face with no harsh shadows.
Style: contemporary expressionist oil painting, clearly visible impasto brushstrokes, bold directional strokes in the hair and fabric, hand-painted canvas texture, a soft Morandi-style earth palette (white, black, sage green, warm skin tones with red-orange accents), museum-grade portrait, masterpiece.
| Model | GPT-Image-2.5 Sunburst | GPT-Image-2 |
|---|---|---|
| Score | 4/5 | 5/5 |
Verdict: the most obvious difference between the two paintings is the direction of the woman's gaze. The prompt says "a young woman looking directly at the viewer over her bare shoulder," and GPT-Image-2 clearly renders this more accurately. Otherwise, neither model has any obvious problems with the woman's appearance, expression, or the overall brushwork and color palette.
Prompt (original text):
A photorealistic overhead shot of an open premium design magazine lying flat on a minimalist light-oak desk, soft natural window light from the left, a matte ceramic coffee cup and a brass pen resting nearby, shallow depth of field, editorial photography.
The magazine’s two-page spread displays a full article, typeset in a clean modern editorial layout (large display title, pull-quote, body columns, page numbers “38 · 39”). ALL TEXT MUST BE RENDERED EXACTLY AS WRITTEN BELOW, correctly spelled, no gibberish, no missing or fused strokes:
LEFT PAGE — header label in small caps gray: “封面专题 · COVER STORY /
深度拆解 · DEEP DIVE” Title block, bold serif Chinese, two lines: 「GPT-Image-2.5 深度拆解： 30亿张图/周的生意，OpenAI 为什么突然把「速度」当主角？」
Directly beneath, English title in lighter italic sans-serif:
“GPT-Image-2.5 Deep Dive: The 3-Billion-Images-a-Week Business — Why Did OpenAI Suddenly Make Speed the Star?”
A red pull-quote strip: 「从『谁画得更惊艳』转向『谁能让创作跟得上思维』」
BODY TEXT in three bilingual column pairs (Chinese paragraph followed by its English translation in smaller gray type):
Paragraph 1: “2026年9月8日，发布GPT-6 Astra仅仅5天后，OpenAI又甩出 ChatGPT Images 2.5：生成延迟比上代砍掉最多50%，但API单价一分不涨。”
Translation 1: “On September 8, 2026 — just five days after GPT-6 Astra — OpenAI unveiled ChatGPT Images 2.5: up to 50% lower generation latency, with API pricing completely unchanged.”
Paragraph 2: “表面看这是一次常规的版本号迭代，实际上藏着一个信号—— 图像模型的军备竞赛，正在从『谁画得更惊艳』转向『谁能让创作跟得上思维』。”
Translation 2: “On the surface, a routine version bump. In fact, a signal: the image-model arms race is shifting from ‘who draws better’ to ‘who keeps up with your thinking’.”
Paragraph 3: “本文将拆解Images 2.5的三大真实升级、Flare与Sunburst的 双子星布局，以及为什么这次『快』比『美』更值得聊。”
Translation 3: “This article breaks down its three real upgrades, the Flare & Sunburst twin-star lineup, and why ‘fast’ beats ‘beautiful’ this time.”
RIGHT PAGE — a large editorial illustration occupies the top half:
a stylized speedometer whose needle points past the red zone, rendered in flat magazine illustration style with the same red accent color as the pull-quote; caption beneath it in tiny type: “图1：速度计 · Fig.1 The Speedometer”. The bottom half continues the body columns. Footer:
“302.AI 基准实验室 · Benchmark Lab · 2026年9月”.
Typography: Chinese in a Song-style serif (思源宋体 feel), English in Inter; crisp, print-quality glyph rendering at legible small sizes; Paper has a subtle matte sheen where the window light crosses it.
| Model | GPT-Image-2.5 Sunburst | GPT-Image-2 |
|---|---|---|
| Score | 5/5 | 3.5/5 |
Verdict: 2.5 wins. On the text itself, neither set of results shows garbled or blurry characters. 2.5 wins on overall realism, with lighting and texture closer to a real photo; the magazine's text and image layout is sensible, and the headers and footers are well placed. GPT-Image-2's magazine repeats the same text on the left and right pages, and parts of the layout do not follow a logical reading order.
Prompt:
Single-page e-commerce product detail page design in an American vintage workwear heritage aesthetic, based on 4 reference images. The denim jacket's design must match the reference photos 100%: the same washed indigo denim, the same spiderweb embroidered patches, the same cinch-back hem, the same silver buttons, and the same model from the reference images. Do not alter the jacket's design or the model's identity.
Layout (vertical e-commerce page, 3:4 aspect ratio):
Top hero section: cream/off-white paper texture background. A large vintage headline in a bold Western slab serif: "SAVAGE DENIM", with a smaller arched banner above it reading "TYPE-1 JACKET", plus a distressed rubber-stamp badge: "$699" inside a brick-red circle. Tagline in a condensed sans-serif: "Heavyweight washed indigo denim. Built to outlive you."
Middle: show the front and back of the jacket, with the front view larger on the left and the back view slightly smaller on the right, both as cutouts with soft drop shadows on the cream background.
Lower half: two model photos side by side in vintage frames with slightly torn paper edges (a front pose and a back pose), with a subtle warm film-grain treatment.
Bottom bar: a deep indigo strip with white type: "14oz Selvedge Denim — Spiderweb Embroidery — Cinch Back — Limited Run", and a small brick-red button: "ADD TO CART".
Color palette: cream #F2EBDC, denim indigo #2E4A66, brick red #B3402A, ink black #1A1A1A, with faded tobacco brown as an accent.
Typography: American vintage signage style: bold slab serif/Western display type for headlines, condensed sans-serif for body text, tracked small caps for labels, all with slight ink bleed and a letterpress texture. Distressed stamps and union-label-style badges as decorative elements.
Quality requirements: professional e-commerce layout, clean grid alignment, print-poster quality, vintage American workwear atmosphere (the feel of 1950s–70s posters), no watermark, no extra products, all English text correctly spelled and legible.
| Model | GPT-Image-2.5 Sunburst | GPT-Image-2 |
|---|---|---|
| Score | 5/5 | 4/5 |
Verdict: both produce accurate, clean cutouts with similar layouts. GPT-Image-2's layout is clearly unbalanced: the SAVAGE DENIM headline is not centered, the left and right margins are uneven, and the front and back views of the jacket differ too much in size. 2.5's poster, with its color scheme, distressed details and rich decorative text, is good enough to deliver as is.
Prompt:
A 2×2 grid on a single canvas, with four equal-sized panels separated by thin white gutters. The subject is identical in all four panels. Character lock: the same Arsenal player, in the red-and-white home kit (red body, white sleeves, "Emirates FLY BETTER" sponsor text, Adidas logo, club crest, Champions League starball badge on the sleeve), white shorts, red socks, long dark hair swept back and flowing slightly, a tattoo on the left arm, an athletic build and fair skin. Scene lock: a night match under floodlights, a lit green pitch, a blurred dark crowd in the stands, orange-and-white LED advertising boards, the official Champions League starball (a white ball with a black star pattern). Photorealistic professional sports photography, a fast shutter freezing the action, shallow depth of field, a slightly low-angle shot, saturated colors.
Use the reference image as Image 1.
Image 2: Based on the reference image: the same Arsenal player has just controlled the ball, which has dropped from his thigh and now rests on the grass directly in front of his standing foot. He stands upright in a balanced athletic stance, both feet on the turf, looking up to read the play ahead, eyes focused into the distance, arms slightly out for balance. Same character, same kit, same floodlit night stadium, same shallow depth of field. Photorealistic sports photography, wide shot, a dynamic yet restrained moment.
Image 3: The same Arsenal player is sprinting forward at full speed, driving forward with the ball, which sits half a step ahead of his right boot; his long dark hair flies back with the speed, his body leaning forward in a driving run. Behind him, a Chelsea defender in a royal-blue kit with white accents stretches out in full pursuit, his arm reaching for the player's shoulder, slightly out of focus. Same floodlit night stadium and blurred crowd, motion blur in the background, focus locked sharply on the Arsenal player. Photorealistic sports photography, a tracking-shot feel, low angle.
Image 4: The same Arsenal player has shaken off the defender and is shooting, in a classic shooting pose: the left standing leg planted beside the ball, the right leg swinging through to strike it, arms out for balance, eyes locked on the target. The Champions League starball has just left his boot, with slight motion blur. The blue-shirted Chelsea defender is a step behind, off balance, his sliding tackle too late, partly blurred. Photorealistic sports photography, a dramatic low-angle shot, a fast shutter freezing the moment, with fine motion blur on the ball and the swinging leg.
| Model | GPT-Image-2.5 Sunburst | GPT-Image-2 |
|---|---|---|
| Score | 4.5/5 | 4/5 |
Verdict: straight to the flaws in each set. 2.5's problems are in the bottom two images, both of which show turf visibly flying up. That makes no sense whether the player is dribbling past a defender or shooting (only a sliding tackle would kick up turf). The shooting motion in Image 4 is also off: the player should be looking in the direction of the pass or shot, not down at his feet, and the ball shows an implausible afterimage. GPT-Image-2's problem is in the top-right image, where the player has both feet off the ground and seems to hover in the air; its Image 4 has the same gaze-direction problem. Beyond that, both models are close to flawless on character consistency and the logic of the movements.
Original image:
Prompt: Change Lightning McQueen's red body to baby blue, and change the red on the wheel hubs to the same yellow as the lightning bolt on the body; remove the car body in the lower-right corner and its shadow.
| Model | GPT-Image-2.5 Sunburst |
|---|---|
| Score | 4.5/5 |
Verdict: the only problem is that the number 95 on the rear of the car was not cleaned up properly, leaving some of the original car's red.
Prompt: Invert the colors of the yellow Pirelli billboards (red background, yellow text); change the black P Zero billboards to green Rolex billboards; add a TV broadcast helicopter to the blue sky in the upper right.
| Model | GPT-Image-2.5 Sunburst |
|---|---|
| Score | 3/5 |
Verdict: all three edits have problems. First, the Pirelli ads on the grandstand roof disappeared and were all replaced with Rolex. Second, the sharpness of the Pirelli ad text at the bottom changed, going from blurry in the original to sharp. Third, the added helicopter is too sharp, so the focus is implausible.
Prompt: Change the text "TRACK VEHICLE" on the black side skirt of the blue car to the Chinese name for Lightning McQueen; add a Red Bull F1 car driven by Verstappen at the spot marked by the green frame.
| Model | GPT-Image-2.5 Sunburst |
|---|---|
| Score | 2.5/5 |
Verdict: this one looks very fake. The F1 car is too close to McQueen and is heading to the right, as if about to crash into him, which is unrealistic. Its tires show no sense of motion and look static, and its body is too sharp, again a focus problem.
After eight sets of tests, the most direct impression is that there is improvement, but it is limited. GPT-Image-2 had already held the SOTA position for a long time, and its capabilities were already top-tier across the board.
OpenAI itself barely mentioned image quality in this launch. The only quality-related line in the announcement was "more natural lighting and richer textures." There were no eye-popping benchmarks and no slogans about breakthroughs.
Here are the Elo scores from the Artificial Analysis Image Arena:
| Version | Elo score | Gain |
|---|---|---|
| GPT-Image-1.5 | 1102 | — |
| GPT-Image-2 | 1171 | +69 points (+6.3%) |
| GPT-Image-2.5 Flare | 1187 | +16 points (+1.4%) |
From 1.5 to 2.0, a 6.3% jump was enough to delight every user, because the gains in image quality and other capabilities were plain to see. This time it is 1.4%: progress is still there, but only in small change.
This is not because OpenAI is not trying hard enough. The race for image quality and realism is itself nearing the finish line. Look at what today's models can do: take prompts thousands of words long, render text accurately and clearly, make precise edits exactly where you point, and stay consistent across multiple rounds. Capabilities that were unthinkable a year ago are now standard among the leading models.
When a model's remaining problems are at the level of "every character is right, but the layout doesn't feel magazine-like enough," the floor for image models has been set, and the ceiling now lies in a higher dimension of aesthetics. In that dimension, Midjourney is still a name you cannot get around.
This is the era of diminishing returns for image generation models: every 1% gain in quality brings users less and less surprise, while moving a leaderboard score by another 10 points may cost exponentially more in research and development. The Elo curve is flattening not because vendors are coasting, but because the ceiling is right overhead.
With image quality hard to push much further, OpenAI's answer is speed.
In the real world, nobody needs a model to produce a single jaw-dropping image that wows the internet. What creators need is the loop of "idea, image, dislike, revise, revise again." Within that loop, going from 90 to 95 points in quality is barely noticeable, but cutting the wait from 100 seconds to 30 seconds is something you feel immediately.
That makes the speed-focused GPT-Image-2.5 Flare the most practical model of this generation:
Faster generation is not just a better experience; it changes the workflow itself. Time costs plunge, more attempts fit into the same amount of time, and creative density doubles.
Source: 302.AI
On September 22, Anthropic released Claude Opus 5.5, pitching intelligence on par with its own flagship Fable 5.1 at 40% less cost than Opus 5. At $4/$20, it is priced attractively among today's flagship models.
OpenAI had no intention of letting its rival own the headlines. The same day, it launched GPT-6 Sol and GPT-6 Luna and went straight at price:
Anthropic had just touted being "40% cheaper than before," and OpenAI answered the same day with an even lower price tag. Two launches on one day, head to head.
With GPT-6, OpenAI has stopped stacking suffixes onto version numbers and switched to astronomical names:
| Model | Positioning | API price (per million tokens, input/output) |
|---|---|---|
| GPT-6 Astra | Flagship, released September 3 | $10 / $50 |
| GPT-6 Sol | Workhorse, this release | $2 / $10 (previous gen $4/$20, down 50%) |
| GPT-5.6 Terra | Mid-tier (no GPT-6 version yet) | $2 / $12 |
| GPT-6 Luna | Lightweight, high-volume, this release | $0.10 / $0.50 (previous gen $0.20/$1.20, down 50%) |
Two details are worth noting:
OpenAI's messaging this time was notably restrained: no talk of AGI, no talk of revolution. Third-party evaluator Artificial Analysis went further, saying in its headline that the models halve prices but "barely move the needle" on performance.
On raw intelligence scores, Sol and Luna indeed set no new records. But measured by intelligence per dollar, they stand out.
According to OpenAI, GPT-6 Luna at high settings matches GPT-5.6 Sol at one-hundredth of the cost.
On Zapier's AutomationBench, which covers real business workflows across sales, marketing, finance and HR:
Note: percentages are end-to-end task completion rates; 33.2% means roughly 33 out of 100 tasks were completed correctly from start to finish.
In internal evaluations based on real user correction data, Sol's error rate was cut in half, bringing its reliability close to Astra's. A workhorse model catching up with the flagship on hallucination rates matters more for everyday use than any coding benchmark.
On the Artificial Analysis leaderboard, GPT-6 Sol and GPT-6 Luna rank 14th and 41st respectively.
The following side-by-side tests pit GPT-6 Sol against GPT-5.6 Terra, the mid-tier workhorse of the 5.6 generation, on the same prompts to see whether the new generation is a meaningful step up.
| Model | Context window | Official price (per million tokens, input/output) |
|---|---|---|
| gpt-6-sol | 1M | $2 / $10 |
| gpt-5.6-terra | 1M | $2 / $12 |
This evaluation focuses on logic, math, coding, multimodal and human-intuition questions. It is not an authoritative test of specialized frontier domains; the aim is to track how models are evolving and to inform model selection.
Coding cases are rated on the following scale:
GPT-6 Sol posted a weighted total of 38.40, ranking 8th on the leaderboard; GPT-5.6 Terra scored 38.27, ranking 12th. Sol held narrow leads in logic and math (9.75 vs 9.70), human intuition (9.85 vs 9.82) and coding simulation (9.80 vs 9.75), while both scored 9.00 in multimodal.
Prompt: Guess which English word the picture describes.
Answer: CHOCOLATE. The singers represent a choir (CHORUS). Replacing letters 4–6 with COLA gives CHOCOLA; then adding NOTE (the musical note) minus NO gives CHOCOLATE.
Prompt 1: Draw an animated SVG of a tornado sweeping through a village.
Prompt 2: Draw an animated SVG of a horse galloping to a river to drink.
| Criterion | GPT-6 Sol | GPT-5.6 Terra |
|---|---|---|
| Semantic accuracy | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ |
| Compositional complexity | ⭐⭐⭐ | ⭐⭐ |
| Animation quality | ⭐⭐⭐ | ⭐⭐ |
Prompt (original English):
Create a polished 3D endless runner game as a single self-contained HTML file.
The gameplay should be inspired by the familiar mechanics of mobile endless runners: the character automatically runs forward along a multi-lane path, while the player can switch lanes, jump, slide, and dodge obstacles. Add collectible items, increasing speed, score tracking, and game-over/restart states.
Create a completely original setting, characters, environment, and visual identity. Do not copy the visual style, characters, locations, props, UI, or assets of any existing game. Use a stylized 3D aesthetic with a strong sense of depth and motion.
Include:
Smooth third-person camera following the character
Three-lane movement with responsive controls
Jumping, sliding, and obstacle avoidance
Procedurally generated sections so the run can continue
Moving obstacles and varied environmental hazards
Collectibles and score progression
Increasing difficulty and speed
Polished animations, lighting, particles, and camera effects
Keyboard and mouse controls
A clear start screen, HUD, game-over screen, and restart button
The final result should feel like a complete, polished 3D mini-game rather than a technical demo.Strengths:
Weaknesses:
Strengths:
Weaknesses:
| Criterion | GPT-6 Sol | GPT-5.6 Terra |
|---|---|---|
| Gameplay creativity | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ |
| Visual execution | ⭐⭐⭐ | ⭐⭐ |
| Technical implementation | ⭐⭐⭐⭐ | ⭐ |
Prompt (original English):
Create a premium immersive 3D web experience about the history and making of traditional Chinese mooncakes, delivered as a single self-contained HTML file.
Design it as a cinematic digital museum rather than a conventional webpage. The visitor should move through a virtual exhibition space and experience the story through carefully choreographed camera movements, lighting, animations, spatial transitions, and interactive 3D objects.
The experience should include:
Origin & History — a cinematic introduction explaining the origins and cultural evolution of mooncakes through an immersive historical gallery.
Ingredients — interactively explore flour, lotus seeds, egg yolks, nuts, and other ingredients as detailed 3D objects.
Making Process — show the complete process: preparing the filling, making the dough, wrapping, shaping with a wooden mold, baking, and the final mooncake emerging from the oven.
Mooncake Showcase — create several original regional and traditional mooncake variations with different shapes, patterns, fillings, and textures.
Cultural Story — connect mooncakes with the Mid-Autumn Festival, family reunion, the full moon, lanterns, and traditional Chinese symbolism.
Build all visual elements yourself with Three.js and procedural geometry/materials. Do not rely on external images, downloaded 3D models, stock assets, or image textures. Model the mooncakes, ingredients, utensils, exhibition architecture, lanterns, moon, clouds, tables, shelves, and other major visual elements from scratch.
Use a sophisticated visual language: dark museum architecture, warm golden illumination, subtle volumetric atmosphere, realistic materials, soft shadows, reflections, floating particles, and restrained Chinese-inspired visual details.
The transitions should feel like a digital film or high-end interactive museum installation rather than a website slideshow. Use cinematic camera choreography, depth of field, slow reveals, object close-ups, environmental animation, and smooth transitions between exhibition spaces.
Make the experience highly polished and interactive. Users should be able to explore objects, trigger animations, zoom into the mooncake-making process, and move naturally between different exhibition areas.
The final result should feel like an award-winning interactive digital exhibition, with strong storytelling, atmosphere, spatial depth, and visual craftsmanship — not a collection of simple 3D objects placed on a webpage.Strengths:
Weaknesses:
It uses a corridor-style spatial narrative that visitors drag through, a good idea. But poor handling of light and shadow, along with rough animation and modeling, makes it visually unappealing and monotonous overall.
| Criterion | GPT-6 Sol | GPT-5.6 Terra |
|---|---|---|
| Visual execution | ⭐⭐⭐⭐ | ⭐⭐ |
| Narrative and information completeness | ⭐⭐⭐⭐ | ⭐⭐⭐ |
| Interaction and user experience | ⭐⭐⭐⭐ | ⭐⭐ |
The bottom line: among flagship-class models, GPT-6 Sol's overall performance is solid but unremarkable.
Compared with the previous mid-tier workhorse, GPT-5.6 Terra, it does improve across the board. But placed within the full GPT-6 lineup, Sol looks somewhat underwhelming.
In these tests, Sol's multimodal reasoning was mediocre, and it slipped up on some logical inference tasks. By contrast, it handled complex tasks better than its predecessor in the same tier, with greater consistency. Put differently, Sol is better at getting a complex project built than at cracking a single hard reasoning puzzle, where its edge is less clear.
As the GPT-6 family's workhorse, Sol can genuinely do it all, covering code, web pages, visual understanding and complex task orchestration. But among flagship models it leans toward stability and caution, without the kind of leap that leaves peers far behind.
In these cases, Sol did well at complex code generation and 3D web development. The projects it produced already had fairly complete engineering structures and could handle layered requirements such as camera systems, animation, sound and interaction logic. Compared with GPT-5.6 Terra, the improvement in technical implementation is plain to see.
In practice, though, if you want the strongest coding ability above all, Sol does not yet deliver flagship-level dominance. Set against Claude Opus 5.5, released in the same period, Sol still falls behind on some complex coding tasks, and Astra remains the higher-tier first choice. Sol is better suited to development work that involves heavy generation and repeated revisions while keeping costs under control.
In short, its coding advantage comes mostly from the balance it strikes between capability, speed and cost.
Price is also what drew the most attention after GPT-6 Sol launched.
Sol's API price is now $2 input and $10 output per million tokens, matching GPT-5.6 Terra on input and undercutting it on output. On top of that, GPT-6's new caching mechanism further reduces the cost of repeated context.
So even if it does not dazzle on every capability metric, for users who work with agents, code and long-context tasks every day, good-enough capability, consistent performance and lower costs are its biggest advantage. In scenarios like long-running coding agents, complex web page generation and batch jobs, a single run may cost only a few dollars, but run dozens of times a day, the cost gap between models widens quickly.
Seen this way, Sol's positioning is very clear: a highly capable workhorse model you can call at scale without worry.
Price is GPT-6 Sol's biggest selling point in this release. Whether it is worth using depends on the task and the trade-offs, but for Sol, the advantage of costing half as much far outweighs how much more capable it has become.
Source: 302.AI
On September 22, SpaceXAI launched its latest flagship model, Grok 4.7. The plainest line in the announcement is also its main selling point: the same price and speed as Grok 4.6 ($2/$6), with a major upgrade in capability.
The key message of the official comparison table: on scores alone, Grok 4.7 is not the winner, but once price is factored in, it becomes the most cost-effective model.
On CursorBench, for example, Fable 5.1 scores 5.5% higher but costs more than five times as much. On the Harvey legal benchmark, Grok 4.7 scores 19.6% against Fable 5.1's 6.7%, nearly three times the score at a lower price.
The official wording is therefore quite precise: it is not the strongest model, but one "at the frontier in price-performance." No ceiling on intelligence, a ceiling on price: that is Musk's current playbook.
The awkward part is that the flurry of model releases in late September made the comparison table outdated the moment it was published: Grok 4.7's rivals are now GPT-6 Astra and Opus 5.5.
Beyond price, safety was the other area the announcement emphasized. SpaceXAI offered a two-sided set of metrics, measuring both whether dangerous requests are blocked and whether legitimate work gets caught in the crossfire:
A model's guardrails are not judged by how many bad requests it refuses: anyone can refuse every bad request, and pushing that score to 100% means nothing. What matters is how low it keeps false refusals while turning bad requests away. A 96.7% refusal rate with few false positives is exactly what Grok 4.7 is emphasizing.
On the Artificial Analysis leaderboard, Grok 4.7 ranks 16th.
Where do Grok 4.7's real-world productivity limits lie? Below is a head-to-head comparison of Grok 4.7 and GPT-6 Sol on the same set of test questions.
| Model | Context window | Official price (per million tokens, input/output) |
|---|---|---|
| grok-4.7 | 500K | $2 / $6 |
| gpt-6-sol | 1M | — |
This evaluation focuses on logic, math, coding, multimodal and human-intuition questions. It is not an authoritative test of specialized frontier domains; the aim is to track how models are evolving and to inform model selection.
Coding cases are rated on the following scale:
GPT-6 Sol posted a weighted total of 38.40, ranking 8th; Grok 4.7 followed closely with 38.39 in 9th place, while its predecessor Grok 4.6 scored 38.38 and ranked 11th. By category, Grok 4.7 led in coding simulation (9.86 vs 9.80), trailed slightly in logic and math (9.73 vs 9.75) and human intuition (9.80 vs 9.85), and tied with Sol at 9.00 in multimodal.
Prompt: Based on the images below, guess which famous film each one depicts.
Answer:
Prompt 1: Draw an animated SVG of two players in a tennis rally.
Prompt 2: Draw an animated SVG of a horse galloping to a river to drink.
| Criterion | Grok 4.7 | GPT-6 Sol |
|---|---|---|
| Semantic accuracy | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ |
| Compositional complexity | ⭐⭐⭐ | ⭐⭐⭐ |
| Animation quality | ⭐⭐⭐⭐ | ⭐⭐⭐ |
Prompt:
Create a polished 3D endless runner game as a single self-contained HTML file.
The gameplay should be inspired by the familiar mechanics of mobile endless runners: the character automatically runs forward along a multi-lane path, while the player can switch lanes, jump, slide, and dodge obstacles. Add collectible items, increasing speed, score tracking, and game-over/restart states.
Create a completely original setting, characters, environment, and visual identity. Do not copy the visual style, characters, locations, props, UI, or assets of any existing game. Use a stylized 3D aesthetic with a strong sense of depth and motion.
Include:
Smooth third-person camera following the character
Three-lane movement with responsive controls
Jumping, sliding, and obstacle avoidance
Procedurally generated sections so the run can continue
Moving obstacles and varied environmental hazards
Collectibles and score progression
Increasing difficulty and speed
Polished animations, lighting, particles, and camera effects
Keyboard and mouse controls
A clear start screen, HUD, game-over screen, and restart button
The final result should feel like a complete, polished 3D mini-game rather than a technical demo.Strengths:
Weaknesses:
Strengths:
Weaknesses:
| Criterion | Grok 4.7 | GPT-6 Sol |
|---|---|---|
| Gameplay creativity | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ |
| Visual execution | ⭐⭐ | ⭐⭐⭐ |
| Technical implementation | ⭐⭐ | ⭐⭐⭐⭐ |
Prompt:
Create a premium immersive 3D web experience about the history and making of traditional Chinese mooncakes, delivered as a single self-contained HTML file.
Design it as a cinematic digital museum rather than a conventional webpage. The visitor should move through a virtual exhibition space and experience the story through carefully choreographed camera movements, lighting, animations, spatial transitions, and interactive 3D objects.
The experience should include:
Origin & History — a cinematic introduction explaining the origins and cultural evolution of mooncakes through an immersive historical gallery.
Ingredients — interactively explore flour, lotus seeds, egg yolks, nuts, and other ingredients as detailed 3D objects.
Making Process — show the complete process: preparing the filling, making the dough, wrapping, shaping with a wooden mold, baking, and the final mooncake emerging from the oven.
Mooncake Showcase — create several original regional and traditional mooncake variations with different shapes, patterns, fillings, and textures.
Cultural Story — connect mooncakes with the Mid-Autumn Festival, family reunion, the full moon, lanterns, and traditional Chinese symbolism.
Build all visual elements yourself with Three.js and procedural geometry/materials. Do not rely on external images, downloaded 3D models, stock assets, or image textures. Model the mooncakes, ingredients, utensils, exhibition architecture, lanterns, moon, clouds, tables, shelves, and other major visual elements from scratch.
Use a sophisticated visual language: dark museum architecture, warm golden illumination, subtle volumetric atmosphere, realistic materials, soft shadows, reflections, floating particles, and restrained Chinese-inspired visual details.
The transitions should feel like a digital film or high-end interactive museum installation rather than a website slideshow. Use cinematic camera choreography, depth of field, slow reveals, object close-ups, environmental animation, and smooth transitions between exhibition spaces.
Make the experience highly polished and interactive. Users should be able to explore objects, trigger animations, zoom into the mooncake-making process, and move naturally between different exhibition areas.
The final result should feel like an award-winning interactive digital exhibition, with strong storytelling, atmosphere, spatial depth, and visual craftsmanship — not a collection of simple 3D objects placed on a webpage.Strengths:
Weaknesses:
Strengths:
Weaknesses:
| Criterion | Grok 4.7 | GPT-6 Sol |
|---|---|---|
| Visual execution | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ |
| Narrative and content completeness | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ |
| Interaction and user experience | ⭐⭐⭐⭐ | ⭐⭐⭐ |
Any assessment of Grok 4.7 runs into the same contradiction: judged on performance alone, it is not the top scorer in the field; factor in cost, and it sits at the frontier of price-performance. Taking the benchmark leaderboards and multiple rounds of hands-on tests together, Grok 4.7's improvement over Grok 4.6 is real and noticeable, but its weaknesses against top-tier flagships are just as hard to ignore.
Grok 4.7 shows gains on long-running tasks and complex knowledge work. The advantage is most visible on tasks that require the model to work through many steps, keep generating code and hold the overall structure together.
In these tests, Grok 4.7 stood out at breaking down and executing complex requirements, and the digital museum in case 4 was especially striking: in a single pass, the model built a multi-hall structure, camera tours, an interaction system and a complete narrative, producing not a pile of simple 3D objects but an immersive experience with a fair degree of completeness.
Grok 4.7 keeps Grok 4.6's pricing of $2 per million input tokens and $6 per million output tokens, which on unit price puts it in the lowest tier among frontier models. With capabilities that can already compete with the top tier, that price leaves plenty of room to use it. For developers who make heavy use of models and also need some capacity for complex tasks, a low unit price directly lowers the cost of trial and error.
In official benchmarks, Grok 4.7 shows clear gains on CursorBench, Terminal-Bench, EEBench and others, but its scores drop in independent third-party tests and do not translate into a clear lead. It ranks only 16th on the Artificial Analysis overall leaderboard, with an Intelligence Index score of 46, a full tier behind GPT-6 Astra and Fable 5.1.
The hands-on cases also show that while Grok 4.7 is a strong generator on complex projects, it is still some way from genuine engineering optimization. For large HTML/Three.js projects like these, the longer the code, the higher the cost of later maintenance and changes.
In addition, the model's safety guardrails have been iterated on, with good results on biosecurity and cybersecurity benchmarks that balance blocking risky requests against sparing legitimate work. In general conversation and fine-grained instruction following, however, it may occasionally produce verbose output or rigid logic.
Although Grok 4.7's pricing is identical to Grok 4.6's, some community feedback notes that on complex tasks the model produces longer reasoning chains, pushing up output token consumption.
On simple tasks, Grok 4.7's per-token cost advantage is clear; in long, continuously iterated workflows, however, the total bill is not necessarily lower than with high-end models.
Grok 4.7 is not a sweeping upgrade toward an all-round general model; instead, more resources have gone into specialized capabilities such as long-running engineering work and agents. Its capability range cannot be defined by benchmark scores alone, and its potential problems should not be glossed over by a low price tag. When choosing a model, beyond benchmark scores and published prices, real-world factors such as task complexity, number of iterations and token consumption need to be considered to judge whether it fits your use case.
Source: 302.AI
When Anthropic released Opus 5.5 on September 22, many developers were already waiting for Sonnet 5.5 and Haiku 5.5. With Haiku 5.5 arriving on October 8, the Claude 5.5 family is now complete: Opus 5.5 (flagship, $4/$20 per million input/output tokens), Sonnet 5.5 (workhorse, $2/$10) and Haiku 5.5 (lightweight, $0.10/$0.50).
The price drew the most attention: $0.10/$0.50, exactly the same as OpenAI's GPT-6 Luna. The two companies have been competing neck and neck on mid- to high-end models for the past two months, and in the lightweight tier they are no longer even staggering their price tags.
One row in the official benchmark table stands out: on Terminal-Bench 4.0 (multistep command-line tasks), Haiku 4.5 scored 0.0% while Haiku 5.5 scored 39.2%.
The previous Haiku did not complete a single task in this test. That was the norm for small models over the past two years: competent at summarizing, classifying, rewriting and translating, but helpless as soon as a task required long-horizon planning.
Key metrics from Haiku 4.5 to Haiku 5.5:
| Benchmark | Haiku 4.5 | Haiku 5.5 | Gain |
|---|---|---|---|
| GDPval-AA (knowledge work) | 735 | 1620 | 2.2× |
| AA-Briefcase (office tasks) | 614 | 1578 | 2.6× |
| OSWorld 2.1 (computer use) | 15.7% | 72.4% | 4.6× |
| HLE (cross-disciplinary reasoning, no tools) | 10.2% | 45.9% | 4.5× |
| Terminal-Bench (command-line tasks) | 0.0% | 39.2% | From zero |
| Chartography (chart understanding) | 6.4% | 46.4% | 7.2× |
The gains are clear, but it should not be measured against the flagship. Anthropic's chosen reference point is its mid-tier workhorse, Sonnet 5.5: 70.6% on Terminal-Bench and 83.9% on OSWorld.
What these numbers really show is that the capability ceiling of small models has moved from "can't" to "entry level." An OSWorld score of 72.4% means a Haiku-class model can handle real computer-use tasks for the first time; a Terminal-Bench score of 39.2% means it can complete about 40% of tasks that require dozens of consecutive steps.
Anthropic is clear about where Haiku 5.5 fits: fast, repetitive work such as summarization, context compaction, database queries and classification; latency-sensitive scenarios such as live customer support and browser automation, where its speed pays off; and coding as a subagent to Opus or Sonnet.
Anyone who has used agentic coding will recognize the setup: while Opus plans the code architecture, someone has to read documentation, check logs, run tests and write summaries, and having the flagship do all that itself is clearly overkill. Haiku 5.5 is there to handle the legwork. It is the first Haiku-class model to offer the same adaptive thinking and effort levels as the rest of the 5.5 family, so effort can be turned down for simple tasks and up for complex ones. It supports a 1M-token context window and 128k output, with native support for browser-use tools.
On the Artificial Analysis leaderboard, Haiku 5.5 currently ranks 31st overall and first among Flash-class models.
Beyond benchmark scores, what matters more is how models at the same price perform in practice. Below is a head-to-head comparison of Haiku 5.5 and GPT-6 Luna on the same set of test questions.
| Model | Context window | Official price (per million tokens, input/output) |
|---|---|---|
| claude-haiku-5-5 | 1M | ≤ 100K: $0.10 / $0.50; > 100K: $0.50 / $2.50 |
| gpt-6-luna | 1.05M | ≤ 272K: $0.10 / $0.50; > 272K: $0.20 / $0.75 |
This evaluation focuses on logic, math, coding, multimodal and human-intuition questions. It is not an authoritative test of specialized frontier domains; the aim is to track how models are evolving and to inform model selection.
Coding cases are rated on the following scale:
Haiku 5.5 posted a weighted total of 38.17, ranking 16th on the leaderboard; GPT-6 Luna scored 37.54, ranking 25th. Both scored 9.60 in logic and math, while Haiku 5.5 led in human intuition (9.80 vs 9.70), coding simulation (9.77 vs 9.74) and multimodal (9.00 vs 8.50).
Prompt: Find the figure that does not follow the same pattern as the others.
Answer: the blue square. Every shape has a matching copy in a different color, except the blue square.
Prompt 1: Draw an animated SVG of two players in a tennis rally.
Prompt 2: Draw an animated SVG of a horse galloping to a river to drink.
Verdict: Haiku 5.5 is slightly ahead of GPT-6 Luna in both compositional complexity and animation.
| Criterion | Claude Haiku 5.5 | GPT-6 Luna |
|---|---|---|
| Semantic accuracy | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ |
| Compositional complexity | ⭐⭐⭐⭐ | ⭐⭐⭐ |
| Animation quality | ⭐⭐⭐ | ⭐⭐ |
Prompt:
Create a polished 3D endless runner game as a single self-contained HTML file.
The gameplay should be inspired by the familiar mechanics of mobile endless runners: the character automatically runs forward along a multi-lane path, while the player can switch lanes, jump, slide, and dodge obstacles. Add collectible items, increasing speed, score tracking, and game-over/restart states.
Create a completely original setting, characters, environment, and visual identity. Do not copy the visual style, characters, locations, props, UI, or assets of any existing game. Use a stylized 3D aesthetic with a strong sense of depth and motion.
Include:
Smooth third-person camera following the character
Three-lane movement with responsive controls
Jumping, sliding, and obstacle avoidance
Procedurally generated sections so the run can continue
Moving obstacles and varied environmental hazards
Collectibles and score progression
Increasing difficulty and speed
Polished animations, lighting, particles, and camera effects
Keyboard and mouse controls
A clear start screen, HUD, game-over screen, and restart button
The final result should feel like a complete, polished 3D mini-game rather than a technical demo.Strengths:
Weaknesses:
Strengths:
Weaknesses:
| Criterion | Claude Haiku 5.5 | GPT-6 Luna |
|---|---|---|
| Gameplay creativity | ⭐⭐⭐ | ⭐⭐⭐ |
| Visual execution | ⭐⭐⭐ | ⭐⭐⭐ |
| Technical implementation | ⭐⭐⭐⭐ | ⭐⭐ |
Prompt:
Create a polished, interactive editorial website titled "The Making of a Painting", exploring how Vincent van Gogh created The Starry Night.
Design it as an immersive digital art exhibition, not a standard gallery or information page. Use elegant typography, generous whitespace, subtle paper textures, large artwork, and a restrained palette inspired by the painting.
Structure the experience as a visual journey:
Opening: Full-screen artwork, title, artist, year, and subtle background motion.
The Context: Explore the painting's historical background, creation, and Van Gogh's circumstances through archival imagery and concise text.
The Sketch: Present preparatory sketches with hover/zoom interactions and a transition from sketch to finished painting.
Color & Composition: Interactive highlights reveal how colors, stars, moon, village, and cypress tree shape the composition.
The Brushwork: Zoom into swirling brushstrokes, using subtle animation to emphasize texture and movement.
The Finished Painting: Dramatically reveal the complete artwork with essential details and a brief interpretation.
Legacy: Conclude with a visual timeline tracing the painting's cultural influence.
Use scroll-driven transitions, parallax, image reveals, hover effects, zoomable artwork, and smooth typography animations. Keep motion subtle and purposeful, like a curated museum exhibition.
Make the website fully responsive, with coherent storytelling, strong visual hierarchy, polished interactions, and high-quality external images where appropriate. Prioritize visual storytelling, editorial design, and atmosphere over complex UI components.Strengths:
Weaknesses:
Strengths:
Weaknesses:
| Criterion | Claude Haiku 5.5 | GPT-6 Luna |
|---|---|---|
| Visuals and interaction | ⭐⭐⭐⭐ | ⭐⭐⭐ |
| Technical implementation | ⭐⭐⭐⭐ | ⭐⭐⭐ |
| Understanding of user intent | ⭐⭐⭐⭐ | ⭐⭐⭐ |
Measured only against same-tier rivals, Haiku 5.5 turns in a more competitive performance: compared with GPT-6 Luna, it leads overall on the basic questions, and its coding output reflects more mature engineering. Still, the price cut lowers the barrier to entry, not the limits of capability; on complex coding and difficult autonomous tasks, it still trails flagship models.
Haiku 5.5 edges out GPT-6 Luna in both multimodal reasoning and SVG drawing, but coding is where the real gap opens up. On complex tasks such as the 3D runner and the themed website, its advantage lies in engineering detail and completeness: the former has fairly complete object management, input feedback and game-state design, while the latter offers exhibition-grade storytelling, an interactive sketch-to-painting transition and detail magnification. Luna has its own highlights in parts of its visual design and responsive layout, but on overall completeness and implementation quality, Haiku 5.5 is the only model at this price point that can beat Luna head-on.
This is the biggest price cut in Haiku's history, with the starting price just one-tenth of its predecessor's. Pricing comes in two tiers: for requests up to 100,000 tokens (which account for 90% of Haiku 4.5 calls), both input and output prices fall by 90%, to $0.10 per million input tokens and $0.50 per million output tokens. Combined with five effort levels, introduced for the first time, developers can scale reasoning effort to task difficulty. Summarization, classification, information extraction and small coding tasks with clear requirements and acceptance criteria all stand to benefit.
But do the savings hold up under scrutiny? Haiku 5.5 uses a new tokenizer, which may increase token consumption for the same content by about 30%. At high effort and in multi-turn agent calls, extra thinking tokens, repeated executions and accumulated context also push costs up. Once a prompt exceeds 100,000 tokens, input and output prices rise to $0.50 and $2.50 per million tokens respectively. When assessing cost-effectiveness, it is best to factor in end-to-end task success rates, token usage, retry counts and execution time; looking only at API unit prices can easily overstate the savings.
Haiku 5.5's value lies in being "orchestrated," not in "carrying the load." Anthropic recommends using it as a subagent to higher-tier models, which matches its capabilities: the flagship handles decomposition and coordination, while Haiku takes on well-scoped, easily parallelized execution work. Its progress is concentrated in execution: OSWorld rose from 15.7% to 72.4%, and HLE without tools from 10.2% to 45.9%. It is highly capable at execution tasks with clear paths, and summarization, classification, context compaction and subagent work are its home turf.
Its weaknesses surface once a task depends on common sense and world knowledge: performance drops noticeably on tasks that require filling in gaps based on real-world context, a telltale sign of limited world knowledge. The coding gap is the most obvious: 39.2% on Terminal-Bench, versus 70.6% for Sonnet 5.5. For complex multistep coding, cross-file refactoring and long-horizon autonomous planning, Sonnet or Opus should still be the first choice.
Only inside a pipeline directed by a flagship model does Haiku 5.5 truly live up to being cheap, fast and good enough. Overall, this is a substantial small-model upgrade: it has an edge over same-tier rivals, its pricing is highly competitive, and it suits high-frequency, standardized, well-scoped workflows. It is worth using to bring down the cost of part of your workload.
Source: 302.AI
According to Anthropic's official release entry, Claude Code 2.1.303 brings bug fixes and reliability improvements. The collected notes do not list individual fixes.
The official release entry lists bug fixes and reliability improvements without details.
According to ggml, llama.cpp b11721 fuses qwen4exp's SCALE, SIGMOID, SCALE and hc_post chain into one CUDA kernel instead of four. The change ports the Vulkan HC_POST_GATE fusion approach.
The notes describe fusing qwen4exp's HC_POST operation chain into one CUDA kernel.
According to ggml, the llama.cpp b11720 prerelease adds non-contiguous SUM support for CPU, CUDA, Vulkan and WebGPU backends, alongside test cases for SUM operations over non-contiguous views.
The prerelease notes add non-contiguous SUM support across several backends and related tests.
According to vLLM, 0.32.0 adds the default /v1/systemone endpoint for structured decisions and hf tool and reasoning parsers that read a model's response_template. It also expands low-precision KV cache support.
The release notes describe structured decisions, generic parsers and expanded low-precision KV cache support.
IT Home and 36Kr report that Microsoft has withdrawn from Nscale's data center project in Loughton, Essex, where it was the planned anchor tenant. IT Home says no reason was disclosed and neither company had responded to requests for comment.
IT Home reports the Loughton withdrawal, with no reason disclosed and no response from either company.
36Kr reports that Microsoft ended the agreement after being the site's planned anchor tenant.
According to Google's API changelog, Gemini 4 Argon is generally available as gemini-4-argon, targeting complex coding, agentic, mathematics and science tasks.
Google's API changelog announces general availability for Gemini 4 Argon.
According to Google's API changelog, Gemini Nano Banana 2.1 is generally available for image generation and conversational editing. The Nano Banana 2 update improves visual quality, instruction following, multi-turn character consistency and text rendering.
Google announces Nano Banana 2.1 general availability and describes image generation improvements.
According to ggml's release notes, llama.cpp b11716 adds Q1_0 and Q2_0 quantization support to the Hexagon backend and enables Q1_0 weights in the 2D HMX matrix multiplication path.
ggml's release notes list two newly supported Hexagon quantization formats and a matrix multiplication fix.
According to Anthropic, 5% of its shares will be committed to a new Public Benefit Fund, subject to stockholder approval. For every 50% rise in value above a set baseline, the board may add another 2% at its discretion.
Anthropic outlines a fund share allocation subject to stockholder approval.
36Kr and IT Home report Xiaoyi voice controls in WeChat on HarmonyOS for texting contacts and starting voice or video calls. IT Home describes the feature as being tested, with group chats and cloned WeChat apps unsupported.
36Kr reports that Xiaoyi can send WeChat messages and start calls on HarmonyOS.
IT Home describes the feature as being tested and lists limits on group chats and cloned apps.
36Kr and IT Home report that DeepSeek V4.1 Flash has been updated to Flash-1017 across its website, app and API. API access and pricing remain unchanged, with deepseek-flash continuing to serve the new version.
36Kr reports that the website, app and API have switched to Flash-1017.
IT Home details the update, unchanged API pricing and availability of model weights.
According to Google's release notes, the 20261017 nightly prerelease of Gemini CLI 0.66.0 adds a banner saying the CLI is not optimized for Gemini 4 and fixes false update success caused by PATH shadowing.
Google's nightly notes add a Gemini 4 warning and fix update verification and keybinding configuration.
According to OpenAI, all API customers can create custom voices from text descriptions through the API or voice Playground and reuse them in GPT-Live sessions. Voice creation is free; speech generation uses regular GPT-Live session billing.
OpenAI describes access and billing for voices generated from text descriptions.
According to ggml's release entry, llama.cpp version b11701 adds support for Reflection's Beam model. The entry lists this model integration as the release's change.
ggml's b11701 release entry lists support for Reflection's Beam model.
According to Anthropic, Claude Code 2.1.302 adds CLAUDE_CODE_AUTO_MODE_MODEL to select the client-side auto mode classifier's model without changing server-side auto mode. The config page also gains an idle compaction toggle.
Anthropic lists client-side auto mode classifier settings and idle compaction controls.
Anthropic announced Claude research into NIST elliptic curve seed origins. IT Home reports five seeds were traced to a pseudorandom generator initialized with Unix timestamps. This is not an ECDSA break; Anthropic says users need take no action.
Anthropic announces research into ECDSA curve seed origins and says users need take no action.
IT Home reports the timestamp and pseudorandom generator reconstruction and remaining gaps.
The Verge and 36Kr report that Meta removed disputed commercial terms wording on webpage content for AI training. The Verge says experts had interpreted it as potentially allowing news-site scraping; the confirmed change is the wording's removal.
Ars Technica reports terms covering linked website content and concerns from legal experts.
The Verge reports that new terms may permit scraping associated sites for AI training.
The Verge reports the removal and describes the earlier scraping implications as experts' interpretation.
36Kr also reports removal of the disputed wording on webpage content for AI training.
According to GitHub, open source license compliance is generally available to Enterprise Cloud customers, including data residency, for repositories with Code Security enabled. Enterprise rulesets can check new or changed dependency licenses before merging.
GitHub details general availability, merge checks and exceptions for dependency license policies.
According to Hugging Face, Transformers 5.20.1 adds Reflection Beam support and fixes model loading on older torch versions. It also corrects auto mappings and image assignment during HunYuan-VL batched generation.
Hugging Face's 5.20.1 notes list Beam support and model loading fixes.
According to GitHub, the Xcode 27 hosted runner image for Actions leaves public preview for building and testing Apple applications. It runs macOS 27 with the Xcode 27 toolchain and Apple SDKs, on Apple Silicon runners only.
GitHub announces general availability of the Xcode 27 hosted image on arm64 only.
According to GitHub, @GitHub in Slack and Teams can now work across multiple repositories in one conversation, including creating issues, changing code and opening pull requests, or investigating bugs across frontend and backend repositories.
GitHub describes cross-repository issues, pull requests and investigations in Slack and Teams.
According to Anthropic, the Compliance API now offers a beta for Claude Enterprise organizations to retrieve members' saved Instructions for Claude in claude.ai and Global instructions in Cowork.
Anthropic announces beta access to saved user instructions through its Compliance API.
According to Anthropic, Opus 5.5 fast mode no longer needs an access request or beta header. Every usage tier has a default fast mode rate limit, and Azure-hosted Microsoft Foundry deployments now support it.
Anthropic says Opus 5.5 fast mode no longer requires an access request or beta header.
According to OpenAI, Codex 0.163.0 adds preference search across categories and marks settings requiring a restart. Follow-ups can steer an active response immediately; interrupting also stops active Code Mode cells and nested tool calls.
OpenAI's Codex 0.163.0 notes add searchable settings and immediate steering during responses.
According to ggml's b11671 release notes, llama.cpp adds Muse Spark Open support across text, vision and audio paths. The implementation includes its latent-space mixture-of-experts architecture and specific quantization adjustments.
ggml's b11671 release notes add Muse Spark Open support for text, vision and audio.
According to GitHub, Team and Enterprise Cloud users can download Code Quality license usage as CSV. Reports list active licensed committers, repositories receiving their commits in the past 90 days, and whether each is unique to one repository or organization.
GitHub adds CSV exports of Code Quality license usage and active committer details.
According to GitHub, Copilot will retire Claude Opus 4.8, Opus 5 and GPT-5.6 Terra on November 16, 2026, across chat, edits, agent mode and completions. Opus 4.8 remains available to individual subscribers on annual plans.
GitHub lists models retiring on November 16, with an exception for annual individual Opus 4.8 users.
According to Anthropic, Claude Code 2.1.301 adds a maximum wait setting for usage-limit resets in unattended retry mode. It also makes Grep and Glob report search-program failures as errors rather than returning no matches.
Anthropic's Claude Code 2.1.301 updates unattended retries and search failure reporting.
According to Ollama, version 0.40.5 adds cloud decision models through clef:cloud and clef-flash:cloud without local downloads. It also fixes continuing OpenAI-compatible Responses conversations that contain custom tool calls.
Ollama 0.40.5 adds cloud decision models and fixes Responses conversation continuation.
According to OpenAI, the Rosalind Academic Plan offers eligible academic labs sponsored access to GPT-Rosalind and tools to advance life sciences research.
OpenAI announces sponsored GPT-Rosalind access for eligible academic research labs.
IT Home, The Verge and TechCrunch report that Bluesky is expanding its experimental Attie AI platform. Users can prompt it to create lightweight interactive apps such as games, polls, quizzes and dashboards, then share them as standalone websites.
IT Home reports games, polls and dashboards while noting the apps remain lightweight.
The Verge describes prompted creation of small social apps shared as standalone websites.
TechCrunch reports interactive experience creation on the experimental Attie platform.
Google describes an Antigravity workflow that found a novel peptide candidate with predicted activity against respiratory pathogens. IT Home also reports the work; wet-lab validation is ongoing, so the predicted effect is not yet experimentally confirmed.
Google describes an Antigravity workflow that found a peptide candidate now undergoing wet-lab validation.
IT Home reports that the workflow uses ProtNLM to scan uncharacterized proteins in UniProt.
According to GitHub, PR AI Scan is generally available to complement CodeQL coverage. Findings are advisory and cannot fail scanning checks or block merges. It does not run on fork pull requests or those created by Dependabot.
GitHub announces general availability of PR AI Scan and says findings cannot block merges.
NVIDIA releases the open Nemotron 3.5 Super VL and Bolt 74B-A9B models for multimodal agents and local workloads. Both are available now; Bolt E4B 4GB for conversational agents is scheduled for October 23. IT Home also reports the launch.
NVIDIA releases Super VL and Bolt 74B while scheduling Bolt E4B for October 23.
IT Home details Super VL's multimodal capabilities and accompanying weights and training resources.
According to GitHub, agentic autofix is generally available to Code Security and Advanced Security customers on Enterprise Cloud, including data residency. Copilot explores code, proposes fixes and reruns CodeQL before opening a draft pull request for review.
GitHub makes agentic autofix generally available, rerunning CodeQL before opening draft pull requests.
According to GitHub, repository push-event email notifications were retired on October 15, 2026. Recipients previously did not need to consent, allowing the feature to be misused for unwanted messages through GitHub's email infrastructure.
GitHub retires repository push-event emails to reduce spam and abuse.
According to Mistral, Timestral is its first time-series foundation model family, with four sizes from 5 million to 209 million parameters. It supports probabilistic multivariate forecasts, past-only and known-future covariates, and adaptation through Mistral Forge.
Mistral introduces Timestral time-series models in four parameter sizes.
According to Anthropic, beta OpenTelemetry export traces Messages API requests and Claude Managed Agents sessions in observability backends. Claude SDKs emit spans in seven languages; API spans require an OpenTelemetry destination in Claude Console.
According to GitHub, HydraFusion and the new Max option remain in research preview across VS Code, the Copilot app and CLI. The updated model pool and workflows tailor reasoning effort for drafting and review, with on-demand repair and parallel critics.
According to Anthropic, version 2.1.300 splits cache-write telemetry by five-minute and one-hour lifetimes and adds GenAI trace attributes. Mods can draw virtual text outside the submitted prompt. An AWS advisor runs only after one is selected.
According to GitHub's updated announcement, workflow lockfiles are in public preview for Enterprise Cloud, including data residency, Team and Free. They keep runs tied to recorded commits despite moving tags or branches; dependencies still require review.
According to Anthropic, file-mention controls let Claude read paths with tools. Long terminal chats can compact after about 54 idle minutes before cache expiry, with an adjustable threshold. Agent view adds effort and fast-mode settings.
According to Ollama, version 0.40.4 supports terminal decision models for yes/no questions, classification and scoring. It adds llama-server multimodal embeddings, local Ollama models in the launched Codex picker, and loading and speculative-decoding fixes.
According to GitHub, Apps and fine-grained PATs gain repository creation and installation-repository permissions. Creation no longer needs broader administration rights covering deletion and transfer. App and credential management also receive updates.
According to Google's release notes, Gemini CLI 0.64.0 serializes file operations, makes writes atomic and recovers corrupted state from backup. It also fixes headless folder-trust propagation, emergency cancellation and Windows interaction issues.
According to GitHub, Gemini 4 Argon is generally available in Copilot, rolling out to Pro+, Max, Business and Enterprise across supported editors, CLI and apps. Billing is usage-based; Business and Enterprise admins can manage access through model policy.
According to GitHub, secret scanning adds Anthropic and Cloudsmith key detectors, and Salesforce joins its partner program. Repositories with secret scanning enabled block the new Anthropic key types by default; partner credentials in public repositories are reported to issuers.
Google says Argon is available through AI Studio and Gemini platform APIs, in Antigravity for AI Ultra, and in the Gemini app and Search AI Mode for Ultra and Pro. Ars Technica also reports access differences between paid plans.
According to Anthropic, its fifth Asia-Pacific office opens in Singapore alongside an MOU with the Ministry of Digital Development and Information covering AI safety, security and model evaluation. The local team will support business and government customers.
According to GitHub, admins can require fresh managed settings at startup in Copilot CLI, VS Code and the Copilot app. Enabled clients stop if policy cannot be confirmed; otherwise normal caching remains. Sandbox settings also enforce minimum client versions.
According to GitHub, withdrawn Dependabot alerts now emit a withdrawn webhook action with prior identifiers. REST reads or updates return 410 instead of 404. The changes are available on github.com and will be included in Enterprise Server 3.24.
According to GitHub, security overview adds AI Scan findings, remediation rates, affected repositories, trends and autofix adoption. This public preview is for Code Security or Advanced Security customers on github.com; Enterprise Server is not supported.