Model sequential and parallel retrieval stages, first-token time and the remaining generated-token stream.
Compute & memory
RAG pipeline critical-path planner
Model sequential and parallel retrieval stages, first-token time and the remaining generated-token stream.
Example inputs · Replace these with your measurements to build your plan.
Same group runs concurrently; different numbered groups run sequentially. Include all work through the first token.
Method, example and sources
Group stages that are truly independent and run concurrently.
Take the longest duration in each barrier group and sum groups.
Treat that sum as the entered time to first token.
Add (output tokens − 1) divided by subsequent token rate.
Worked example. 40 + max(30,25) + 50 + 200 gives 320 ms to first token. Another 99 tokens at 50 tokens/s takes 1,980 ms: 2,300 ms through the final token.
Stages with the same group run concurrently; groups run sequentially. Include first-token work in the stage ledger.
Time to first token
320ms
Additional token generation
1,980ms
Time through final token
2,300ms
Largest entered stage
prefill and first token
RAG pipeline critical-path planner: detailed result
Execution group
Critical-path duration (ms)
1
40
2
30
3
50
4
200
This is a declared dependency graph with barrier groups, not a general scheduler. Do not put dependent stages in the same group.
Token rate and first-token latency depend on model, context, hardware, batching, software and concurrency; entered numbers are not a GPU benchmark or capacity guarantee.
From the catalog
Servers
Explore related listings, then confirm the exact model requirements against your plan. These listings are not an automatic compatibility or capacity verdict.
A RAG request can spend time rewriting, retrieving, reranking and preparing generation before showing a token. Parallel retrieval stages should contribute their critical path, not the sum of both durations. Assign the same execution group only when work can actually overlap, and include all dependent work through the first generated token.
What to verify next
Measured model and service timings are essential inputs. Context length, batch size, concurrent users, model precision and the serving stack can change both first-token delay and streaming rate. This worksheet does not predict a GPU's performance or evaluate answer quality. Compare representative traces, and preserve dependencies when experimenting with a faster stage so a hypothetical speedup does not erase work that remains necessary.
Worked example
40 + max(30,25) + 50 + 200 gives 320 ms to first token. Another 99 tokens at 50 tokens/s takes 1,980 ms: 2,300 ms through the final token.
