Domestic LLM 'LLM-jp-4 33B' Released — 33.2 Billion Parameters Trained on 11.7 Trillion Tokens (National Institute of Informatics)

Explanation

On August 18, 2026, the LLM Research and Development Center (LLMC) at the National Institute of Informatics (NII) released a new Japanese large language model, "LLM-jp-4 33B."

Two versions were released. One is "llm-jp-4-33b-base," a foundational model that has completed pretraining and intermediate training, and the other is "llm-jp-4-33b-thinking," which adds post-training via supervised fine-tuning (SFT) and direct preference optimization (DPO) on top of the base. The former can be thought of as the basic foundation, and the latter as having been trained to respond in line with human instructions.

The scale is about 33.2 billion parameters. Parameters are the adjustable numerical values inside a model, determined through the training process. The architecture is Dense type (using all parameters every time), with 64 layers, a hidden layer size of 5,120, and 40 attention heads. The context length—the length of text that can be handled at once—is 65,536 tokens.

A total of 11.7 trillion tokens were used for training. A token is a slightly smaller unit than a word; in Japanese, one token corresponds to roughly one to a few characters. The corpus (collection of text) used for training has been made public, but some portions remain undisclosed due to licensing restrictions.

The performance of the post-trained model was measured on four benchmarks: Japanese MT-Bench, English MT-Bench, AnswerCarefully, and llm-jp-instructions. It outperformed previous models in the LLM-jp-4 Thinking series that LLMC had released, on all of them. Note that the comparison here is with the series' own previous models, not with commercial models from overseas.

The license is Apache License 2.0, allowing broad use including commercial applications. The model itself is distributed via Hugging Face, and the DPO dataset used to train the thinking model was also released as "llm-jp-4-33b-thinking-dpo-data."

Releasing not just the model weights but also the training data has significance. For many models released by companies, it is unclear how the internals were built. If the training data is public, it is possible to externally verify why the model produces the answers it does. This difference matters when models are used in research and education settings.

Source: National Institute of Informatics LLM Research and Development Center, "Release of the approximately 33.2-billion-parameter LLM 'LLM-jp-4 33B'" (August 18, 2026)
https://llmc.nii.ac.jp/topics/post-2941/