ISTA DeepSeek-V4.1-Flash Quantization Benchmark Comparison Clarification
ISTA addresses benchmark discrepancies with Unsloth regarding DeepSeek-V4.1-Flash-1017, highlighting the impact of reasoning budget limits.
ISTA has addressed recent benchmark comparisons published by Unsloth that differed from their results for DeepSeek-V4.1-Flash-1017 (763B parameters):
- Discrepancy Source: The performance variance stems from benchmark constraints rather than model quality. Unsloth’s comparison used a strict 32K token reasoning budget, whereas ISTA’s benchmarks allowed up to 128K output tokens.
- Standard Settings: ISTA used a 128K context window and a reasoning effort set to 50.
- Open Collaboration: All code is available on GitHub and papers on arXiv. The team invites independent verification using their exact settings.