Welcome Gemma - Google’s new open LLM

概要

• Googleが軽量なオープンモデル Gemmaを新しく公開
• HuggingFace: https://huggingface.co/models?other=gemma&sort=trending&search=google
• paper: https://storage.googleapis.com/deepmind-media/gemma/gemma-report.pdf
• https://ai.google.dev/gemma?hl=ja

• 種類
• gemma-7b: Base 7B model.
• gemma-7b-it: Instruction fine-tuned version of the base 7B model.
• gemma-2b: Base 2B model.
• gemma-2b-it: Instruction fine-tuned version of the base 2B model.

• 性能



• Gemma-7BはLLama 70Bに匹敵するレベルの性能！

• Model Architecture
• transformer decoderベース
• Multi-Query Attention(2B)
• RoPE Embeddings
• GeGLU Activations
• Normalizer Location