HDD-RoPE: A Dynamic Rotary Positional Embedding Proposal

A researcher proposes HDD-RoPE, a dynamic variant of Rotary Positional Embedding (RoPE), and reports improved validation loss on TinyStories when training a model with this method.

One continuously updated timeline instead of dozens of separate articles. New developments are appended as the story evolves.

1. AnnouncementJun 24, 2026, 06:16 PM 64%

   Researcher introduces HDD-RoPE, a dynamic variant of RoPE, reporting improved validation loss on TinyStories

   A researcher proposes HDD-RoPE, a dynamic variant of Rotary Positional Embedding (RoPE), and reports improved validation loss on TinyStories when training a model with this method.

   Read the full story →