登壇者 Speakers
黒橋 禎夫 KUROHASHI Sadao
LLM-jp/NII-LLMC: Building a Sovereign LLM Ecosystem through Open and Team Science
国立情報学研究所 所長／大規模言語モデル研究開発センター センター長
Director-General, National Institute of Informatics／Director, Recearch and Development Center for Large Language Models (LLMC)
1994年京都大学大学院工学研究科博士課程修了。博士（工学）。2006年4月より京都大学大学院情報学研究科教授。2023年4月より同特定教授および国立情報学研究所長を併任。自然言語処理、知識情報処理の研究に従事。言語処理学会10周年記念論文賞、同20周年記念論文賞、文部科学大臣表彰科学技術賞等を受賞。2024年4月より国立情報学研究所内に大規模言語モデル研究開発センターを設置し、全国の研究者と透明性・信頼性の高い日本語版LLMの構築を目指し研究開発を進めている。
Sadao Kurohashi received a PhD in Electrical Engineering from Kyoto University in 1994. He is currently the Director-General of the National Institute of Informatics, Japan, and a specially appointed professor at the Graduate School of Informatics at Kyoto University. His research interests include natural language processing, knowledge infrastructure, and open science. He received the 10th and 20th anniversary best paper awards from Journal of Natural Language Processing in 2004 and 2014, respectively, 2009 IBM faculty award, and the 2010 NTT DOCOMO mobile science award, and the 2017 Commendation for Science and Technology by the Minister of Education.
Noah Smith
Science of AI and AI for Science
Neural language models with billions of parameters and trained on trillions of words are powering the fastest-growing computing applications in history and generating discussion and debate around the world. Yet most scientists cannot study or improve those state-of-the-art models because the organizations deploying them keep their data and machine learning processes secret. I believe that the path to models that are usable by all, at low cost, customizable for areas of critical need like the sciences, and whose capabilities and limitations are made transparent and understandable, is radically open development, with academic and not-for-profit researchers empowered to do reproducible science. In this talk, I’ll discuss some of the work our team is doing to radically open up the science of language modeling and make it possible to explore new scientific questions and democratize control of the future of this fascinating and important technology.
The work I’ll present was led by a large team at the Allen Institute for Artificial Intelligence in Seattle, with collaboration from the Paul G. Allen School at the University of Washington and various kinds of support and coordination from many organizations, including the Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University, AMD, CSC – IT Center for Science (Finland), Databricks, Together.ai, and the National AI Research Resource Pilot. In August, the team was awarded a $75M mid-scale research infrastructure grant from the National Science Foundation, with additional support from NVIDIA, enabling continued work for five years.
Professor, University of Washington ／ the Allen Institute for AI, Senior Director of NLP Research
Noah A. Smith is a researcher in natural language processing and machine learning, serving as the Amazon Professor at the University of Washington and Senior Director of NLP Research at the Allen Institute for AI. He co-directs the OLMo open language modeling initiative and is the PI of the NSF- and NVIDIA-supported project “Open Multimodal AI Infrastructure to Accelerate Science.” His current work spans language, music, and AI research methodology, with a strong emphasis on mentoring—his former mentees now hold faculty and leadership roles worldwide. Smith is a Fellow of the Association for Computational Linguistics and has received numerous awards for research and innovation.
Patrick Lewis
RAG: Past, Present, Future
In approximately 2020, improvements in information retrieval systems and natural language generation technology led to the first truly convincing generative systems with non-parametric memory storage components. ‘Retrieval-Augmented Generation’ or ‘RAG’– a term that initially referred to a narrow, specific modelling technique – has since become part of the popular AI lexicon and has spawned an extremely large array of techniques, systems and applications and created a market sector worth billions of dollars.
In this talk, I will reflect on progress in retrieval-augmented generation over the past 5 years. We’ll examine what has changed, what’s remained true, and what new challenges and important problems are emerging.
Senior Director of Agents, Cohere
Dr. Patrick Lewis is an AI Research Scientist and the Senior Director of Agentic AI at Cohere, where he leads the RAG (Retrieval-Augmented Generation), tool-use, and agent teams. Based in London, his work focuses on the intersection of information retrieval (IR), agentic AI and large language models (LLMs), aiming to build more powerful, efficient, and robust models to power useful agentic systems.
Prior to Cohere, Patrick was a Research Scientist at Meta AI’s FAIR lab. He completed his PhD at University College London (UCL), split between FAIR and UCL, supervised by Professor Sebastian Riedel and Professor Pontus Stenetorp, and holds a Master’s in Natural Sciences from the University of Cambridge. His research has significantly advanced techniques like Retrieval-Augmented Language Models, and he co-authored the seminal 2020 paper introducing RAG. This method enables AI models to answer queries using external sources, reducing hallucinations and incorporating up-to-date information, an approach that has become ubiquitous in modern LLM-powered applications.
横田 理央 YOKOTA Rio
MoE型LLMの論理推論性能とスパーシティの関係について
Optimal Sparsity of Mixure-of-Experts Language Models for Reasoning Tasks
経験的なスケーリング法則は大規模言語モデル（LLM）の進化を牽引してきたが、モデルアーキテクチャやデータパイプラインが変更されるたびにその係数は変動する。最先端システムで標準となった混合エキスパートモデル（MoE）は、現在のDenseモデル研究が見落としている新たな疎性を導入する。本研究では、MoEのスパーシティが二つの異なる能力領域（記憶能力と論理推論能力）に与える影響を調査する。固定された計算リソース予算下で、総パラメータ数・アクティブパラメータ数・Top-kルーティングを変化させたMoEモデル群を探索することで、事前学習損失と下流タスク精度を分離分析した結果、二つの原則が明らかになった。第一に、アクティブFLOPsについては訓練損失が同一でもアクティブ計算量が多いモデルほど推論精度が高い。次に、パラメータあたりの総トークン数（TPP）については記憶課題はパラメータ数増加で改善する一方、推論課題は最適TPPで効果を発揮し、推論がデータ集約的であることを示唆する。強化学習（GRPO）でもテスト時スケーリングにおいてもこの傾向は維持される。したがって、最適MoEスパース性はアクティブFLOPsとTPPの兼ね合いによって決定され、新たな計算量最適スケーリングが必要になる。
Empirical scaling laws have driven the evolution of large language models (LLMs), yet their coefficients shift whenever the model architecture or data pipeline changes. Mixture-of-Experts (MoE) models, now standard in state-of-the-art systems, introduce a new sparsity dimension that current dense-model frontiers overlook. We investigate how MoE sparsity influences two distinct capability regimes: memorization skills and reasoning skills. By training MoE families that vary total parameters, active parameters, and top-k routing under fixed compute budgets, we disentangle pre-training loss from downstream accuracy. Our results reveal two principles. First, Active FLOPs: models with identical training loss but greater active compute achieve higher reasoning accuracy. Second, Total tokens per parameter (TPP): memorization tasks improve with more parameters, while reasoning tasks benefit from optimal TPP, indicating that reasoning is data-hungry. Neither reinforcement learning post-training (GRPO) nor increased test-time compute alters these trends. We therefore argue that optimal MoE sparsity must be determined jointly by active FLOPs and TPP, revising the classical picture of compute-optimal scaling.
東京科学大学 教授
Professor, Institute of Science Tokyo
2009年慶應義塾大学理工学研究科博士課程修了、同年Bristol大学ポスドク研究員、2011年Boston大学ポスドク研究員、2012年King Abdullah University of Science and Technology常勤研究員、2015年より東京工業大学学術国際情報センター准教授、2023年より東京工業大学学術国際情報センター教授、2024年より東京科学大学総合研究院スーパーコンピューティング研究センター教授、現在に至る。高性能計算、大規模深層学習に関する研究に従事。博士（工学）
Rio Yokota is a Professor at the Supercomputing Research Center, Institute of Integrated Research, Institute of Science Tokyo. He also leads the AI for Science Foundation Model Research Team at RIKEN Center for Computational Science. His research interests lie at the intersection of high performance computing, machine learning, and linear algebra. He has been optimizing algorithms on GPUs since 2007, and was part of a team that received the Gordon Bell prize in 2009 using the first GPU supercomputer. More recently, he has been leading distributed training efforts on Japanese supercomputers such as ABCI, TSUBAME, and Fugaku. He is the co-developer of the Japanese LLM Swallow, and LLM-jp. He is also involved in the organization of multinational collaborations such as ADAC and TPC.
宮尾 祐介 MIYAO Yusuke
大規模言語モデルの日本語理解能力の評価と分析
Assessing the Japanese Language Understanding Capabilities of Large Language Models
本講演では、日本語に強い大規模言語モデルの開発を目指す LLM-jpプロジェクトにおける言語モデルの評価と分析の研究を紹介する。近年の大規模言語モデルは、多言語を含む膨大なテキストデータを学習することで、多言語理解能力を獲得している。しかし、学習データの多くは英語で構成されており、日本語のように言語的・文化的特徴が大きく異なる言語に対して、その性能を適切に評価・分析することが求められている。本講演では、日本語ベンチマークの構築方法や評価結果の分析を通して、日本語における大規模言語モデルの評価の現状と課題を概説する。
This talk presents research on the evaluation and analysis of large language models conducted as part of the LLM-jp Project. Recent large language models have acquired multilingual understanding capabilities by learning from massive text datasets covering a wide range of languages. However, as most training data consist of English text, it is crucial to properly evaluate and analyze model performance in languages that differ linguistically and culturally from English, such as Japanese. This talk provides an overview of the current status and challenges in evaluating large language models for Japanese, with a focus on the construction of Japanese benchmarks and the analysis of evaluation results.
東京大学 教授
Professor, The University of Tokyo
2000年東京大学大学院理学系研究科修士課程修了。2001年より同大学にて助手、2007年より助教。2006年同大学大学院にて博士号 （情報理工学）取得。2010年より国立情報学研究所准教授、2018年より東京大学教授。構文解析、意味解析などの自然言語処理基盤技術とその応用の研究に従事。
Yusuke Miyao received his Ph.D. from the University of Tokyo in 2006. He has been Assistant Professor at the University of Tokyo from 2001 to 2010, Associate Professor at National Institute of Informatics since 2010, and Professor at the University of Tokyo from 2018. He has been engaged in research on natural language processing, in particular on syntactic parsing and its applications.
関根 聡 SEKINE Satoshi
日本語LLMの安全性構築について
Building Safety on Japanese LLMs
日本語LLMの安全性を構築するために、複数のデータを構築し、評価基準を設計、人手評価を実施してきた。安全性データとしては、バイヤス・差別・ヘイト・反公序良俗、AIとの対話リスク、情報漏洩、悪用、誤情報・偽情報を対象にしたAnswerCarefulluという1800件のデータを構築し、広く利用されている。他にも、具体的な誤情報・偽情報データ、Jailbreakデータ、フィルタリングのための安全性に問題があるWebページなどのデータを構築、公開している。安全性の評価は、有用性との密接な関係があり、その両方を考慮した評価基準を設計し、12システムの出力を人手で評価し、その結果をllm-as-a-judgeの自動評価と比較した。最後に、セキュリテイーやエージェントAIを含むより広い安全性のベンチマークをAll Japan/One Teamで構築するプロジェクトについても紹介する。
To construct the safety functionalities on Japanese LLMs, we have built multiple data sets, designed evaluation criteria, and conducted manual evaluations. For safety data, we have compiled a dataset called “AnswerCarefully”, which contains 1,800 items of data covering bias, discrimination, hate, anti-public order and morals, risks of interaction with AI, information leaks, abuse, and misinformation. This data was relieased in September 2024, and has been widely used in LLM development community. We have also compiled and published data on specific misinformation and disinformation, jailbreak data, and web pages with safety issues for filtering purpose. Safety evaluation is closely related to usability, so we designed evaluation criteria that took both into account. Then we manually evaluated the output of 12 models using the criteria, and compared the results with the automated evaluation by llm-as-a-judge. Finally, we will introduce a project to build a broader safety benchmark including security, agent AI and other topics as an All Japan/One Team effort.
国立情報学研究所 特任教授
Project Professor, National Institute of Informatics
1992年英国マンチェスター大学計算言語学部修士号。1998年ニューヨーク大学コンピューターサイエンス学部博士号。ニューヨーク大学研究准教授の他、パナソニック、SONY、楽天、理化学研究所AIPなどでの研究職を歴任。専門は自然言語処理。特に知識構築、情報抽出、固有表現抽出の研究に従事。現在は国立情報学研究所大規模言語モデル研究開発センター特任教授。他に株式会社いちから、ランゲージクラフト、複数の企業の技術顧問などを兼任。
Prof. Sekine received MSc in Computational Linguistics from the University of Manchester in the UK in 1992 and a PhD in Computer Science from New York University in 1998. In addition to being a research associate professor at New York University, he has held research positions at Panasonic, Sony, Rakuten, and RIKEN AIP, among others. His specialty is natural language processing, with a particular focus on knowledge construction, information extraction, and so on. He is currently a professor at the Large Language Model Research and Development Center at the National Institute of Informatics. He also serves as a technical advisor to Ichikara Co., Ltd., Language Craft, and several other companies.
河原 大輔 KAWAHARA Daisuke
日本語に強い大規模言語モデルの開発のためのコーパス構築
Corpus Construction for High-Performance Japanese Large Language Models
日本語に強い大規模言語モデルを構築するためには、大規模かつ高品質な日本語コーパスを用いて事前学習を行う必要がある。我々は、このようなコーパスを構築するために、様々な言語資源からテキストを収集し、処理している。これまでに利用している主な言語資源は、ウェブクロールデータ、Wikipedia、科研費研究課題概要、特許文書、法律文書、国会議事録などである。これらから日本語テキストを抽出、フィルタリング、重複除去などを行うことによってコーパスを構築し、LLM-jpモデルの構築に用いるとともに、一般に公開している。本講演では、これまでのコーパス構築において直面した課題や解決策について紹介するとともに、今後のコーパス開拓の方向性について議論する。
To develop large language models with strong performance in Japanese, it is essential to conduct pre-training on large-scale and high-quality Japanese corpora. To this end, we have been collecting and processing texts from various language resources. The main language resources we have used so far include web-crawled data, Wikipedia, summaries of KAKEN research projects, patent documents, legal texts, and the minutes of the National Diet. By extracting, filtering, and deduplicating Japanese text from these resources, we have constructed corpora that are used for building the LLM-jp models and are also publicly available. In this talk, I will introduce the challenges we have faced and the solutions we have developed in corpus construction, and discuss future directions for expanding and improving Japanese corpora.
早稲田大学 教授
Professor, Waseda University
2002年京都大学大学院博士課程単位取得認定退学。2005年同修了。博士(情報学)。東京大学大学院情報理工学系研究科学術研究支援員、情報通信研究機構主任研究員、京都大学大学院情報学研究科准教授を経て、2020年より早稲田大学理工学術院教授。専門は知識獲得、言語理解評価を中心とする自然言語処理。
Daisuke Kawahara received his Ph.D. in Informatics from Kyoto University in 2005, after completing the doctoral program requirements in 2002. He was a research associate at the Graduate School of Information Science and Technology, the University of Tokyo, a Senior Researcher at the National Institute of Information and Communications Technology (NICT), and an associate professor at the Graduate School of Informatics, Kyoto University. Since 2020, he has been a professor at the Faculty of Science and Engineering, Waseda University. His research interests include natural language processing, particularly knowledge acquisition and language understanding evaluation.
鈴木 潤 SUZUKI Jun
大規模言語モデルの事前学習と中間学習
Pre-training and Mid-training for Large Language Models
大規模言語モデルを構築する工程では、多段階の学習パイプラインを用いることが主流となっている。大まかには、事前学習・中間学習・事後学習の3段階の学習パイプラインで大規模言語モデルを構築することが多い。各段階には、それぞれ異なる目的が設定されており、例えば、事前学習では、大量の学習データを用いてより広範な世界知識や言語知識を獲得することを目指しており、中間学習では、数学やコード生成の性能、あるいは、より長い文脈を扱えるような機能拡張といった特定の分野の性能向上や機能向上を目指す。そのため、これまでの研究で得られた知見に基づいて、各段階で必要・有益と考えられる学習データを主に人手で用意し、各段階に適した学習手法とハイパーパラメータを設定して学習を行う。学習データの準備などには組織ごとの知見が反映されることも多く、目指す方向性は同じでも細部は構築されるモデルによって異なることが一般的と言える。本講演では、LLM-jpにおける大規模言語モデル構築の取り組みについて、主に事前学習から中間学習にかけての取り組みと、そこで得られた経験や知見を簡単に共有する。
When building large language models these days, a multi-stage training pipeline is a standard approach. At a high level, the pipeline comprises three stages: pre-training, mid-training, and post-training, each with distinct objectives and design choices. For example, pre-training focuses on broad coverage and scale, exposing the model to a diverse mixture of text to build general linguistic and world knowledge. Mid-training then sharpens the model’s abilities through targeted data and objectives (for example, reasoning in math and code, and adaptation to longer contexts). At each stage, appropriate datasets are curated primarily manually, based on findings from previous studies. In addition, learning algorithms and their hyperparameters are selected for each stage. The details of the training datasets and methods can differ across models, since data curation practices may vary across organizations. In this talk, I will outline our current approach, focusing mainly on the pre-training and mid-training stages, and share the experiences and insights we have gained from our activities.
東北大学 教授
Professor, Tohoku University
2001年から2018年まで日本電信電話株式会社コミュニケーション科学基礎研究所研究員（特別研究員）。2018年より東北大学大学院情報科学研究科准教授、2020年より同大学データ駆動科学・AI教育研究センター教授．2020年から2022年までGoogle LLC Visiting Researcher（クロスアポイントメント）。2023年、言語AI研究センター新設とともにセンター長に就任（現職）。博士（工学）。
Jun Suzuki is a professor at Tohoku University and, since 2023, director of its Language AI Research Center. He joined Tohoku University in 2018 as an associate professor in the Graduate School of Information Sciences and became a professor at the Data-Driven Science and AI Education and Research Center in 2020. From 2020 to 2022, he held a cross-appointment as a visiting researcher at Google. Earlier, from 2001 to 2018, he was a researcher and later a distinguished researcher at NTT Communication Science Laboratories. He holds a Ph.D. in Engineering.
菅原 朔 SUGAWARA Saku
評価・チューニングWGにおける研究活動
Research in Evaluation and Tuning Working Group
国立情報学研究所 助教
Assistant Professor, National Institute of Informatics
2020年3月東京大学大学院情報理工学系研究科博士課程修。2020年4月より現職。自然言語処理・計算言語学の研究に従事。 博士（情報理工学）。
Sugawara received a Ph.D. in Information Science and Technology (The University of Tokyo, 2020). Since April 2020, he is Assistant Professor at National Institute of Informatics, working on natural language processing and computational linguistics.
岡崎 直観 OKAZAKI Naoaki
日本語の画像言語モデルの構築
Building Japanese Vision-Language Models
大規模言語モデル（LLM）の発展により、テキストだけでなく画像で入力された情報を処理できる画像言語モデル (Vision-Language Model; VLM) が登場している。ところが、既存のVLMが日本の風景や文化、習慣、日本語をどのくらい理解できているか、もし理解が不十分であるとしたらどのように改善すればよいのか、明らかになっていない。本講演では、LLM研究開発センターで進められているVLM構築の取り組みを紹介する。具体的には、Common CrawlやWikipediaなどから画像と日本語テキストが対になった訓練データの構築、VLM構築のための日本語指示チューニングデータ、VLMの能力を評価するための評価基盤、および評価のためのデータセット構築などの取り組みを紹介する。
With the advancement of large language models (LLMs), vision-language models (VLMs) have emerged that can process not only textual but also visual inputs. However, it remains unclear: how well existing VLMs understand Japanese landscapes, culture, customs, and the Japanese language itself; and, if their understanding is limited, how such models can be improved. In this talk, I will introduce the ongoing efforts at the LLM Research and Development Center (LLMC) to build Japanese VLMs. Specifically, I will present our work on constructing training data consisting of image–Japanese text pairs collected from sources such as Common Crawl and Wikipedia, developing Japanese instruction-tuning datasets for VLM training, establishing evaluation frameworks to assess VLM capabilities, and creating benchmark datasets for systematic evaluation.
東京科学大学 教授
Professor, Institute of Science Tokyo
東京科学大学（旧・東京工業大学、改称）情報理工学院 教授
2007年東京大学大学院情報理工学系研究科博士課程修了。博士（情報理工学）。東京大学大学院情報理工学系研究科・特任研究員、東北大学大学院情報科学研究科准教授を経て、2017年8月より現職。自然言語処理の研究に従事。言語処理学会理事、日本ディープラーニング協会（JDLA）理事。平成28年度科学技術分野の文部科学大臣表彰若手科学者賞、第15回船井学術賞、2016年度マイクロソフト情報学研究賞などを受賞。
Naoaki Okazaki, currently a professor at the School of Computing, Institute of Science Tokyo, earned his PhD in 2007 from the Graduate School of Information Science and Technology at the University of Tokyo. His early post-doctoral career included a research position at the same institution. Subsequently, he became an associate professor at the Graduate School of Information Sciences at Tohoku University. He specializes in Natural Language Processing and Artificial Intelligence. His career is marked by numerous awards, such as the Young Scientists’ Prize, the Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology. In addition, he was honored with the 2016 Microsoft Research Award on Information Processing.
尾形 哲也 OGATA Tetsuya
実環境インタラクションWG – Physical AIへの取り組み –
早稲田大学 教授
Professor, Waseda University
1993年早稲田大学理工学部機械工学科卒業、1997年日本学術振興会特別研究員（DC2）、1999年早稲田大学理工学部助手、2001年理化学研究所脳科学総合研究センター研究員、2003年京都大学大学院情報学研究科講師、2005年同准教授を経て、2012年より早稲田大学理工学術院基幹理工学部表現工学科教授。博士（工学）。2009年-2015年JSTさきがけ領域研究員。また2017年より産業総合技術研究所人工知能研究センター特定フェロー。2013年から2014年日本ロボット学会理事。2016年から2018年人工知能学会理事などを歴任。2020年より早稲田大学次世代ロボット研究機構AIロボット研究所所長。2024年より国立情報学研究所大規模言語モデル研究開発センター客員教授。2025年よりAIロボット協会理事長。2025年よりJST CREST「実環境知能システム」領域研究総括。2021年IEEE ICRA2021 Best Paper Award In Cognitive Science、2023年文部科学大臣表彰科学技術賞（研究部門）などを受賞。
Tetsuya Ogata received the B.S., M.S., and D.E. degrees in mechanical engineering from Waseda University, Tokyo, Japan, in 1993, 1995, and 2000, respectively. He was a Research Associate with Waseda University from 1999 to 2001. From 2001 to 2003, he was a Research Scientist with the RIKEN Brain Science Institute, Saitama, Japan. From 2003 to 2012, he was an Associate Professor at the Graduate School of Informatics, Kyoto University, Kyoto, Japan. Since 2012, he has been a Professor with the Faculty of Science and Engineering at Waseda University. From 2009 to 2015, he was a JST (Japan Science and Technology Agency) PREST Researcher. Since 2017, he has been a Joint-appointed Fellow with the Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology, Tokyo, and a visiting professor with the Research and Development Center for Large Language Models, National Institute of Informatics (NII). He has been the director of the Institute of AI and Robotics at Waseda University since 2020, the chairperson of the AI Robot Association (AIRoA), and the Research supervisor of JST CREST “Fundamentals and Core Technologies for Embodied AI” since 2025.
東中 竜一郎 HIGASHINAKA Ryuichiro
日本語音声対話モデルの構築
Development of Japanese Spoken Dialogue Models
大規模言語モデルとの対話が，人間同士の自然な音声対話として実現されれば，そのポテンシャルは飛躍的に拡大するものと期待される．LLM-jpにおける対話WGでは，話しながら聞くという双方向性を備えた音声対話モデルに着目し，対話データの収集とモデルの構築を推進してきた．透明性の確保を重視しつつ，日本語音声対話モデルの開発を段階的に進めるとともに，対話システムにおいて重要なTheory of Mind（心の理論）ベンチマークの構築にも並行して取り組んでいる．本講演においては，LLM-jp対話WGの活動概要，収集したデータセット，およびモデル構築を通じて得られた知見について報告する．
If dialogue with large language models can be realized through natural spoken interaction similar to human conversation, their potential is expected to expand significantly. The Dialogue Working Group at LLM-jp has focused on full-duplex spoken dialogue models capable of speaking while listening, and has been advancing the collection of dialogue data and model development. While emphasizing transparency, we are progressively developing Japanese spoken dialogue models and concurrently working on the construction of Theory of Mind benchmarks, which are important for dialogue systems. In this presentation, we will report on the activities of the LLM-jp Dialogue Working Group, the datasets we have collected, and the insights gained through model development.
名古屋大学 教授
Professor, Nagoya University
2001年慶應義塾大学大学院政策・メディア研究科博士前期課程，2008年博士後期課程修了．2001年日本電信電話株式会社入社．2020年より，名古屋大学大学院情報学研究科教授．対話システムの研究に従事．博士（学術）．
Ryuichiro Higashinaka completed his Master’s program at the Graduate School of Media and Governance, Keio University in 2001, and completed his Doctoral program and received his Ph.D. in 2008. He joined NTT in 2001. Since 2020, he has been a Professor at the Graduate School of Informatics, Nagoya University. He is engaged in research on dialogue systems.
大関 洋平 OSEKI Yohei
大規模言語モデルの原理解明
Elucidating the Principles of Large Language Models
本講演では、大規模言語モデルの原理解明について概観します。具体的には、解釈性、汎化性、効率性など大規模言語モデルの問題点を指摘した上で、大規模言語モデル研究開発センターで構築した大規模言語モデルの原理解明をミッションとする「原理解明WG」の背景・目的を説明するのと同時に、動作原理と学習原理の2つに関する原理解明WGの成果を報告します。
In this talk, I will review the elucidation of the principles of large language models (LLMs). Specifically, after pointing out the problems with LLMs like interpretability, generalizability, and efficiency, I will explain the background/purpose of the “Principles Elucidation” WG whose mission is to elucidate the principles of LLMs developed at the Research and Development Center for Large Language Models, and present the achievements of the WG on working/learning principles of LLMs
東京大学 准教授
Associate Professor, The University of Tokyo
東京大学大学院総合文化研究科准教授、国立情報学研究所大規模言語モデル研究開発センター特任研究員。2012~2013年にマサチューセッツ大学アマースト校言語学科で訪問学生、2013~2018年にニューヨーク大学言語学科でPh.D.を取得。2018~2020年に早稲田大学理工学術院助教、2020~2024年に東京大学大学院総合文化研究科講師を経て、2024年から現職。計算言語学、認知科学の研究に従事。
Yohei Oseki is an Associate Professor in the Department of Language and Information Sciences at the University of Tokyo, and a Project Researcher in the Research and Development Center for Large Language Models at the National Institute of Informatics (NII). Before joining the University of Tokyo, he was an Assistant Professor in the Faculty of Science and Engineering at Waseda University in 2018~2020, and received a Ph.D. from New York University in 2013~2018 and visited the University of Massachusetts Amherst in 2012~2013. His research integrates computational linguistics with cognitive sciences.