| Presentation | 2026-03-25 Optimizing the Ordering in the Chain of Thought for Arithmetic Asuki Hosokawa, Koei Naito, Yuta Sato, Hiroshi Kera, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Auto-regressive generation with Transformers enables sequential generation of reasoning processes as token sequences, and has been applied to tasks involving multi-step reasoning such as arithmetic. The order of intermediate steps (chain of thought) generated by the decoder can affect learning difficulty. Sato et al. identified learning-friendly orders for arithmetic tasks through order search, while pointing out that relaxing permutation matrices to real values and mixing them can cause information leakage from future tokens, and proposed a search-based approach. This study re-examines the approach of relaxing permutation matrices to real values that Sato et al. avoided. Specifically, we verify the case of learning permutations without information leakage using the Sinkhorn algorithm and Straight Through Estimator (STE). We experiment on two arithmetic tasks with high order sensitivity and investigate the effectiveness and limitations of this method. The results showed that the proposed optimization improves the accuracy of partial sequence prediction but rather reduces the complete match rate (the proportion of all tokens matching exactly). This reaffirms that learning the order through gradient-based optimization is difficult with the proposed method. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Transformer / Autoregressive Generation / Order Optimization / Arithmetic Learning |
| Paper # | IBISML2025-49 |
| Date of Issue | 2026-03-17 (IBISML) |
| Conference Information | |
| Committee | PRMU / IPSJ-CVIM / IBISML / ITE-SIP |
|---|---|
| Conference Date | 2026/3/24(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Hideo Saito(Keio Univ.) / / Toshihiro Kamishima(independent researcher) / Mitsugu Kakuta(Nippon Sport Science Univ.) |
| Vice Chair | Masato Ishii(Sony AI) / Masashi Nishiyama(Tottori Univ.) / / Atsuyoshi Nakamura(Hokkaido Univ.) / Koji Tsuda(Univ. of Tokyo) / Dan Mikami(Kogakuin Univ.) |
| Secretary | Masato Ishii(Tokyo Denki Univ..) / Masashi Nishiyama(Denso IT Laboratory.) / / Atsuyoshi Nakamura(Osaka Univ.) / Koji Tsuda(Univ.of Tokyo) / Dan Mikami |
| Assistant | Rei Kawakami(Science Tokyo) / Yoshihiko Mochizuki(SIT) / / Mahito Sugiyama(NII) / SAKUMA JUN(Science Tokyo) |
| Paper Information | |
| Registration To | Technical Committee on Pattern Recognition and Media Understanding / Special Interest Group on Computer Vision and Image Media / Technical Committee on Information-Based Induction Sciences and Machine Learning / Technical Group on Sport Information Processing |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Optimizing the Ordering in the Chain of Thought for Arithmetic |
| Sub Title (in English) | |
| Keyword(1) | Transformer |
| Keyword(2) | Autoregressive Generation |
| Keyword(3) | Order Optimization |
| Keyword(4) | Arithmetic Learning |
| 1st Author's Name | Asuki Hosokawa |
| 1st Author's Affiliation | Chiba University(Chiba U) |
| 2nd Author's Name | Koei Naito |
| 2nd Author's Affiliation | Chiba University(Chiba U) |
| 3rd Author's Name | Yuta Sato |
| 3rd Author's Affiliation | Chiba University(Chiba U) |
| 4th Author's Name | Hiroshi Kera |
| 4th Author's Affiliation | Chiba University(Chiba U) |
| Date | 2026-03-25 |
| Paper # | IBISML2025-49 |
| Volume (vol) | vol.125 |
| Number (no) | IBISML-425 |
| Page | pp.pp.60-66(IBISML), |
| #Pages | 7 |
| Date of Issue | 2026-03-17 (IBISML) |