Issues

• qwen3-0.6b fine_tuning 64k OOM

  jquesnelle/yarn#67 · Filter by author dgzxx-2000 opened on Jun 9, 2025
  1

• ModuleNotFoundError: No module named 'scaled_rope.modeling_qwen3_yarn'

  jquesnelle/yarn#66 · Filter by author dgzxx-2000 opened on Jun 9, 2025

• Have you tried Rope_base = 500k in YaRN?

  jquesnelle/yarn#64 · Filter by author hannlp opened on Sep 4, 2024

• For models with other architectures, such as Qwen family, how to find the best \alpha, \beta and \sqrt{1/t} parameters?

  jquesnelle/yarn#63 · Filter by author ki-ljl opened on Jul 3, 2024

• the detail of code implements.

  jquesnelle/yarn#62 · Filter by author cooper12121 opened on Jun 14, 2024

• Question related to _yarn_linear_ramp_mask

  jquesnelle/yarn#60 · Filter by author chizhang118 opened on Apr 9, 2024
  1

• OOM error of distributed training on 80GB GPUs with Mistral-7b

  jquesnelle/yarn#59 · Filter by author TracyPlus opened on Apr 8, 2024
  3

• cannot connect to hugging face

  jquesnelle/yarn#58 · Filter by author RENNY-Jenius opened on Mar 29, 2024

• Why the updated cache is initialized with seqlen=256?

  jquesnelle/yarn#57 · Filter by author pUmpKin-Co opened on Mar 27, 2024

• Trying to set a tensor of shape torch.Size([257, 1024]) in "weight" (which has shape torch.Size([1226, 1024])), this look incorrect

  jquesnelle/yarn#56 · Filter by author Litong-sTs opened on Mar 18, 2024

• Can we run the replication of the results，8 * 80 A100

  jquesnelle/yarn#55 · Filter by author zhanglv0209 opened on Mar 15, 2024
  1

• How should I proceed with conducting an evaluation for lm-evaluation-harness?

  jquesnelle/yarn#54 · Filter by author yuri-son opened on Mar 15, 2024

• Questions about DynamicNTK

  jquesnelle/yarn#53 · Filter by author wutong4012 opened on Mar 14, 2024

• An OOM error occurred while computing the perplexity of 128k Proofpoint documents with a maximum token count set to 128k.

  jquesnelle/yarn#52 · Filter by author HIT-cwh opened on Mar 5, 2024
  1

• Phi 2

  jquesnelle/yarn#51 · Filter by author fakerybakery opened on Feb 25, 2024

• Could this repository be used for sft based on YaRN?

  jquesnelle/yarn#50 · Filter by author Zheng-Jay opened on Feb 19, 2024

• OOM on two 80GB GPUs

  jquesnelle/yarn#49 · Filter by author kyleliang919 opened on Jan 9, 2024
  6

• Unexpected larger perplexity on PG19

  jquesnelle/yarn#48 · Filter by author Yiyi-philosophy opened on Dec 10, 2023
  1

• Running Error

  jquesnelle/yarn#44 · Filter by author wangyang-stu opened on Nov 22, 2023
  2

• RoPE scaling config confusing

  jquesnelle/yarn#40 · Filter by author tattrongvu opened on Nov 15, 2023

• Inquiry Regarding Evaluation Metrics in Your Paper

  jquesnelle/yarn#39 · Filter by author teslacool opened on Nov 13, 2023
  2

• context length and dataset size

  jquesnelle/yarn#38 · Filter by author shossain opened on Nov 6, 2023

• Hardware equipments and training time?

  jquesnelle/yarn#37 · Filter by author zhoumengbo opened on Nov 6, 2023

• What is the purpose of finetuned parameter in LlamaDynamicYaRNScaledRotaryEmbedding?

  jquesnelle/yarn#36 · Filter by author fahadh4ilyas opened on Nov 6, 2023

• Mistral-train error on deepspeed config

  jquesnelle/yarn#35 · Filter by author xiechengmude opened on Nov 2, 2023
  1