<link>https://chiphan1110.github.io/</link></image><item><title>Getting Started</h1> <article> <h1>Getting Started</h1> <p>Sat, 17 Feb 2024 00:00:00 +0000</p> <h2 id="quick-start-from-template">Quick Start from Template</h2> <div class="hb-steps"> <h3 id="create-a-site">Create a site</h3> <p> </p> <h3 id="configure-your-new-site">Configure your new site</h3> <p> </p> <h3 id="add-your-content">Add your content</h3> <p> </p> <h3 id="publish-your-site">Publish your site</h3> <p>Your site will automatically publish ~1-5 minutes after you commit (save) changes to files in your GitHub repository.</p> </div> <h2 id="next">Next</h2> <p>Let’s customize your new site:</p> <div class="hb-cards mt-4 grid gap-4 not-prose" style="--hb-cols: 1;"> <a class="hb-card group"href="../guide/project-structure" > <span class="hb-card-title p-4"> <svg style="height: 1em; width: 1em;" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M15.75 17.25v3.375c0 .621-.504 1.125-1.125 1.125h-9.75a1.125 1.125 0 0 1-1.125-1.125V7.875c0-.621.504-1.125 1.125-1.125H6.75a9 9 0 0 1 1.5.124m7.5 10.376h3.375c.621 0 1.125-.504 1.125-1.125V11.25c0-4.46-3.243-8.161-7.5-8.876a9 9 0 0 0-1.5-.124H9.375c-.621 0-1.125.504-1.125 1.125v3.5m7.5 10.375H9.375a1.125 1.125 0 0 1-1.125-1.125v-9.25m12 6.625v-1.875a3.375 3.375 0 0 0-3.375-3.375h-1.5a1.125 1.125 0 0 1-1.125-1.125v-1.5a3.375 3.375 0 0 0-3.375-3.375H9.75"/></svg>Project Structure</span></a> <a class="hb-card group"href="../guide/configuration" > <span class="hb-card-title p-4"> <svg style="height: 1em; width: 1em;" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M6 13.5V3.75m0 9.75a1.5 1.5 0 0 1 0 3m0-3a1.5 1.5 0 0 0 0 3m0 3.75V16.5m12-3V3.75m0 9.75a1.5 1.5 0 0 1 0 3m0-3a1.5 1.5 0 0 0 0 3m0 3.75V16.5m-6-9V3.75m0 3.75a1.5 1.5 0 0 1 0 3m0-3a1.5 1.5 0 0 0 0 3m0 9.75V10.5"/></svg>Configuration</span></a> <a class="hb-card group"href="../guide/formatting" > <span class="hb-card-title p-4"> <svg style="height: 1em; width: 1em;" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M15.75 17.25v3.375c0 .621-.504 1.125-1.125 1.125h-9.75a1.125 1.125 0 0 1-1.125-1.125V7.875c0-.621.504-1.125 1.125-1.125H6.75a9 9 0 0 1 1.5.124m7.5 10.376h3.375c.621 0 1.125-.504 1.125-1.125V11.25c0-4.46-3.243-8.161-7.5-8.876a9 9 0 0 0-1.5-.124H9.375c-.621 0-1.125.504-1.125 1.125v3.5m7.5 10.375H9.375a1.125 1.125 0 0 1-1.125-1.125v-9.25m12 6.625v-1.875a3.375 3.375 0 0 0-3.375-3.375h-1.5a1.125 1.125 0 0 1-1.125-1.125v-1.5a3.375 3.375 0 0 0-3.375-3.375H9.75"/></svg>Create content</span></a> </div> </article> <article> <h1>Customizing Hugo</h1> <p>Mon, 01 Jan 0001 00:00:00 +0000</p> <p>View the full docs at </p> </article> <article> <h1>Project Structure</h1> <p>Mon, 01 Jan 0001 00:00:00 +0000</p> <h2 id="folder-structure">Folder Structure</h2> <p>There are <strong>4 main folders for Hugo-based sites</strong>:</p> <ul> <li><code>content/</code> for your Markdown-formatted content files (homepage, etc.) <ul> <li><code>_index.md</code> the homepage (<strong>Hugo requires that the homepage and archive pages have an underscore prefix</strong>)</li> </ul> </li> <li><code>assets/</code> <ul> <li><code>media/</code> for your media files (images, videos) <ul> <li><code>icons/custom/</code> upload any custom SVG icons you want to use</li> </ul> </li> </ul> </li> <li><code>config/_default/</code> for your site configuration files <ul> <li><code>hugo.yaml</code> to configure Hugo (site title, URL, Hugo options, setup per-folder page features)</li> <li><code>module.yaml</code> to install or uninstall Hugo themes and plugins</li> <li><code>params.yaml</code> to configure Hugo Blox options (SEO, analytics, site features)</li> <li><code>menus.yaml</code> to configure your menu links (if the menu is enabled in <code>params.yaml</code>)</li> <li><code>languages.yaml</code> to configure your site’s language or to set language-specific options in a multilingual site</li> </ul> </li> <li><code>static/uploads/</code> for any files you want visitors to download, such as a PDF</li> <li><code>go.mod</code> sets the version of Hugo themes/plugins which your site uses</li> </ul> <h2 id="hugo-file-naming-convention">Hugo File Naming Convention</h2> <p>Hugo gives us two options to name standard page files: as <code>TITLE/index.md</code> or <code>TITLE.md</code> where <code>TITLE</code> is your page name.</p> <p>The page name should be lowercase and using hyphens (<code>-</code>) instead of spaces.</p> <p>Both approaches result in the same output, so you can choose your preferred approach to naming and organizing files. A benefit to the folder-based approach is that all your page’s files (such as images) are self-contained within the page’s folder, so it’s more portable if you wish to share the original Markdown page with someone.</p> <p>The homepage is a special case as <strong>Hugo requires the homepage and listing pages to be named</strong> <code>_index.md</code>.</p> <h2 id="docs-navigation">Docs Navigation</h2> <p>The docs navigation is automatically generated based on the content in the <code>docs/</code> folder and is sorted alphabetically.</p> <p>The order of pages can be changed by adding the <code>weight</code> parameter in the front matter of your Markdown files.</p> <p>In the example below, the <code>example.md</code> page will appear before the <code>test.md</code> page as it has a lower <code>weight</code>:</p> <p>Page <code>example.md</code>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nn">---</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="nt">title</span><span class="p">:</span><span class="w"> </span><span class="l">My Example</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="nt">weight</span><span class="p">:</span><span class="w"> </span><span class="m">1</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="nn">---</span><span class="w"> </span></span></span></code></pre></div><p>Page <code>test.md</code>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nn">---</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="nt">title</span><span class="p">:</span><span class="w"> </span><span class="l">My Test</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="nt">weight</span><span class="p">:</span><span class="w"> </span><span class="m">2</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="nn">---</span><span class="w"> </span></span></span></code></pre></div> </article> <article> <h1>Configuration</h1> <p>Mon, 01 Jan 0001 00:00:00 +0000</p> <p>The configuration of your site can be found in <code>config/_default/</code>.</p> <h2 id="full-documentation">Full Documentation</h2> <p>See </p> <h2 id="navigation">Navigation</h2> <h3 id="menu">Menu</h3> <p>See </p> <h2 id="left-sidebar">Left Sidebar</h2> <p>Links are automatically generated from the structure of your content directory. Simply add a folder to nest a page.</p> <h3 id="extra-links">Extra Links</h3> <p>Additional links can be added under the <code>sidebar</code> section of your <code>config/_default/menus.yaml</code>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">menu</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">sidebar</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s2">"Need help?"</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">params</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">separator</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">weight</span><span class="p">:</span><span class="w"> </span><span class="m">1</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s2">"A page"</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">pageRef</span><span class="p">:</span><span class="w"> </span><span class="s2">"/page-filename-here"</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">weight</span><span class="p">:</span><span class="w"> </span><span class="m">2</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s2">"An external link ↗"</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">url</span><span class="p">:</span><span class="w"> </span><span class="s2">"https://hugoblox.com"</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">weight</span><span class="p">:</span><span class="w"> </span><span class="m">3</span><span class="w"> </span></span></span></code></pre></div><h2 id="right-sidebar">Right Sidebar</h2> <p>A table of contents is automatically generated from the headings your Markdown file.</p> <p>It can optionally be disabled by setting <code>toc: false</code> in the front matter of a page:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nn">---</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="nt">title</span><span class="p">:</span><span class="w"> </span><span class="l">My Page</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="nt">toc</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="nn">---</span><span class="w"> </span></span></span></code></pre></div> </article> <article> <h1>Enhancing Pathological VLMs with Cross-scale Reasoning</h1> <p>Wed, 01 Jul 2026 00:00:00 +0000</p> </article> <article> <h1>PathScale-R1: Cross-scale Reasoning for Pathological Image Analysis</h1> <p>Wed, 01 Jul 2026 00:00:00 +0000</p> </article> <article> <h1>Never-smoker Under-representation in Training Data Reduces AI Performance for EGFR Mutation Prediction in Singapore Lung Adenocarcinoma Patients</h1> <p>Thu, 01 Jan 2026 00:00:00 +0000</p> <p>This manuscript is listed in the CV as under review for <strong>npj Precision Oncology</strong>.</p> </article> <article> <h1>Notes on the Foundations of Reinforcement Learning and LLM Post-Training (Part 2)</h1> <p>Tue, 02 Sep 2025 00:00:00 +0000</p> <details class="print:hidden xl:hidden" open> <summary>Table of Contents</summary> <div class="text-sm"> <nav id="TableOfContents"> <ul> <li><a href="#before-we-begin">Before We Begin</a></li> <li><a href="#1-language-generation-as-an-rl-problem">1. Language Generation as an RL Problem</a> <ul> <li><a href="#problem-formulation">Problem Formulation</a></li> <li><a href="#the-optimization-objective">The Optimization Objective</a></li> </ul> </li> <li><a href="#2-learning-from-human-preferences">2. Learning from Human Preferences</a> <ul> <li><a href="#why-learn-a-reward-function">Why Learn a Reward Function?</a></li> <li><a href="#preference-learning-step-by-step">Preference Learning, Step by Step</a></li> </ul> </li> <li><a href="#3-the-models-in-the-rlhf-pipeline">3. The Models in the RLHF Pipeline</a></li> <li><a href="#4-sft-and-reward-model-training">4. SFT and Reward Model Training</a> <ul> <li><a href="#supervised-fine-tuning-sft">Supervised Fine-Tuning (SFT)</a></li> <li><a href="#reward-model">Reward Model</a></li> </ul> </li> <li><a href="#5-ppo-training">5. PPO Training</a> <ul> <li><a href="#actor-model">Actor Model</a></li> <li><a href="#critic-model">Critic Model</a></li> <li><a href="#reference-model-and-token-rewards">Reference Model and Token Rewards</a></li> <li><a href="#practical-training-considerations">Practical Training Considerations</a></li> <li><a href="#open-challenges">Open Challenges</a></li> </ul> </li> <li><a href="#6-group-relative-policy-optimization-grpo">6. Group Relative Policy Optimization (GRPO)</a> <ul> <li><a href="#motivation">Motivation</a></li> <li><a href="#which-component-changes">Which Component Changes?</a></li> <li><a href="#advantage-function-estimation-in-grpo">Advantage Function Estimation in GRPO</a></li> <li><a href="#a-learning-question-how-can-one-advantage-train-every-token">A Learning Question: How Can One Advantage Train Every Token?</a></li> <li><a href="#the-grpo-objective-and-loss">The GRPO Objective and Loss</a></li> <li><a href="#actor-training">Actor Training</a></li> </ul> </li> <li><a href="#7-ppo-and-grpo-at-a-glance">7. PPO and GRPO at a Glance</a></li> <li><a href="#key-takeaways">Key Takeaways</a></li> <li><a href="#acknowledgements">Acknowledgements</a></li> <li><a href="#references">References</a> <ul> <li><a href="#rl-foundations">RL Foundations</a></li> <li><a href="#policy-gradient-papers">Policy Gradient Papers</a></li> <li><a href="#rlhf">RLHF</a></li> <li><a href="#ppo-and-grpo-explanations">PPO and GRPO Explanations</a></li> </ul> </li> </ul> </nav> </div> </details> <h2 id="before-we-begin">Before We Begin</h2> <p> introduces policy gradients, value functions, actor–critic methods, and PPO. Here, I connect those ideas to language-model post-training: how a response becomes a trajectory, how preferences become rewards, and how PPO and GRPO update the model.</p> <p>The reading path is <strong>language generation as RL → preference learning → SFT and reward models → PPO training → GRPO</strong>. If the notation becomes unfamiliar, return to Part 1’s concepts and PPO sections.</p> <h2 id="1-language-generation-as-an-rl-problem">1. Language Generation as an RL Problem</h2> <h3 id="problem-formulation">Problem Formulation</h3> <p>Given a prompt, a language model generates a response one token at a time. This autoregressive process gives us a concrete mapping to the RL concepts from Part 1.</p> <table> <thead> <tr> <th>RL concept</th> <th>Language-model interpretation</th> </tr> </thead> <tbody> <tr> <td>State $s_t$</td> <td>The prompt and all tokens generated so far</td> </tr> <tr> <td>Action $a_t$</td> <td>The next token</td> </tr> <tr> <td>Policy $\pi_\theta(a_t\mid s_t)$</td> <td>A probability distribution over the vocabulary</td> </tr> <tr> <td>Transition</td> <td>Append the selected token: $s_{t+1}=[s_t,a_t]$</td> </tr> <tr> <td>Reward $r_t$</td> <td>Feedback assigned after generating a token</td> </tr> <tr> <td>Value $V(s_t)$</td> <td>Expected future return from the current prefix</td> </tr> </tbody> </table> <p>The initial state is the prompt. For a VLM, the conditioning context can also include visual input. The state transition is deterministic once the next token has been selected, even though token sampling is stochastic.</p> <h3 id="the-optimization-objective">The Optimization Objective</h3> <p>Let $x$ denote a prompt and $y$ a response. With a response-level reward $r(x,y)$, the objective is</p> $$ \max_\theta\;\mathbb{E}_{x\sim D,\,y\sim\pi_\theta(\cdot\mid x)}[r(x,y)]. $$<p>A KL penalty discourages the policy from drifting too far from a reference model. A regularized objective can be written as</p> $$ \max_\theta\;\mathbb{E}_{x\sim D}\left[ \mathbb{E}_{y\sim\pi_\theta(\cdot\mid x)}[r(x,y)] -\beta D_{\mathrm{KL}}\left(\pi_\theta(\cdot\mid x)\,\|\,\pi_{\mathrm{ref}}(\cdot\mid x)\right) \right]. $$<p>Equivalently, using the sampled log-probability ratio inside the expectation:</p> $$ \max_\theta\;\mathbb{E}_{x\sim D,\,y\sim\pi_\theta(\cdot\mid x)}\left[ r(x,y)-\beta\log\frac{\pi_\theta(y\mid x)}{\pi_{\mathrm{ref}}(y\mid x)} \right]. $$<p>Here $\beta$ controls the strength of the penalty. This fixed reference policy has a different role from the old rollout policy used in PPO’s clipping ratio.</p> <h2 id="2-learning-from-human-preferences">2. Learning from Human Preferences</h2> <h3 id="why-learn-a-reward-function">Why Learn a Reward Function?</h3> <p>For tasks such as conversation or summarization, it is difficult to write a reward function that captures what people want. Comparing examples can be easier than assigning an absolute numerical score. RLHF uses those comparisons to learn a reward model, then optimizes a policy against that model.</p> <h3 id="preference-learning-step-by-step">Preference Learning, Step by Step</h3> <h4 id="1-collect-preference-data">1. Collect Preference Data</h4> <p>Annotators compare two segments of agent behavior, $\sigma^1$ and $\sigma^2$. A segment is a sequence of state–action pairs:</p> $$ \sigma^i=((s_0^i,a_0^i),\ldots,(s_{k-1}^i,a_{k-1}^i)). $$<p>Use a target $p$ of $1$ when the first segment is preferred, $0$ when the second is preferred, and $0.5$ for a tie. This notation makes the label convention explicit.</p> <h4 id="2-fit-a-reward-model">2. Fit a Reward Model</h4> <p>The model assigns a score to each segment by summing predicted per-step rewards:</p> $$ S_{\hat r}(\sigma)=\sum_t\hat r(s_t,a_t). $$<p>Under a Bradley–Terry preference model, the probability that the first segment is preferred is</p> $$ \hat P(\sigma^1\succ\sigma^2) =\frac{\exp S_{\hat r}(\sigma^1)} {\exp S_{\hat r}(\sigma^1)+\exp S_{\hat r}(\sigma^2)}. $$<p>This is a softmax over two scores: a larger predicted cumulative reward gives a higher preference probability. The language-model example later uses a scalar score for a whole response rather than a sum of learned per-token rewards.</p> <p>Let $\hat p=\hat P(\sigma^1\succ\sigma^2)$. Fit the predictions to the preference labels with cross-entropy:</p> $$ \mathcal{L}(\hat r) =-\mathbb{E}_{(\sigma^1,\sigma^2,p)\sim D} \left[p\log\hat p+(1-p)\log(1-\hat p)\right]. $$<p>For strictly ordered pairs, denote the preferred segment by $\sigma^+$ and the rejected segment by $\sigma^-$. The loss becomes</p> $$ \begin{aligned} \mathcal{L}(\hat r) &=-\mathbb{E}\left[\log\hat P(\sigma^+\succ\sigma^-)\right]\\ &=-\mathbb{E}\left[ \log\frac{\exp S_{\hat r}(\sigma^+)} {\exp S_{\hat r}(\sigma^+)+\exp S_{\hat r}(\sigma^-)} \right]. \end{aligned} $$<h4 id="3-optimize-the-policy">3. Optimize the Policy</h4> <p>Once the reward function is learned, an RL algorithm uses it as feedback:</p> <ol> <li>The policy generates a response or trajectory.</li> <li>The reward model assigns a score.</li> <li>A policy-optimization algorithm such as PPO uses the score to update the policy.</li> <li>The updated policy generates new samples for the next round.</li> </ol> <p>The alternates preference collection, reward learning, and policy learning. Collecting new feedback can help cover behavior that was absent from earlier data, but human annotation is expensive. Automated preference labeling, including RLAIF, is another approach to obtaining feedback.</p> <p>The learned reward remains a proxy for the preferences represented in its training data. Improving that score and improving behavior should be evaluated separately.</p> <h2 id="3-the-models-in-the-rlhf-pipeline">3. The Models in the RLHF Pipeline</h2> <p>The PPO-based pipeline discussed here has four roles: <strong>actor, critic, reward model, and reference model</strong>. The SFT checkpoint supplies an initial policy and commonly initializes other components.</p> <table> <thead> <tr> <th>Component</th> <th>Role during RL</th> <th>Updated during this stage?</th> </tr> </thead> <tbody> <tr> <td>Actor / policy</td> <td>Generates responses and learns from rewards</td> <td>Yes</td> </tr> <tr> <td>Critic / value model</td> <td>Predicts returns for response prefixes</td> <td>Yes</td> </tr> <tr> <td>Reward model</td> <td>Scores completed responses</td> <td>Frozen in this setup</td> </tr> <tr> <td>Reference model</td> <td>Supplies a fixed distribution for the KL penalty</td> <td>No</td> </tr> </tbody> </table> <p>The SFT model is a pretrained language model further trained on demonstrations. It provides a useful starting point before preference-based RL.</p> <p> <figure id="figure-figure-1-overview-of-the-rlhf-training-pipeline"> <div class="flex justify-center "> <div class="w-full" ></div> </div><figcaption> Figure 1. Overview of the RLHF training pipeline. </figcaption></figure> </p> <p>The overall sequence is:</p> <ol> <li><strong>Prepare the SFT checkpoint</strong> using prompt–response demonstrations.</li> <li><strong>Train the reward model</strong> using preferred and rejected response pairs.</li> <li><strong>Generate rollouts</strong> with the actor and score the sampled tokens under the actor and reference policies.</li> <li><strong>Build the reward signal</strong> from response scores and the KL penalty.</li> <li><strong>Estimate advantages and return targets</strong> using rewards and critic predictions.</li> <li><strong>Update actor and critic</strong> with their respective losses, then collect another rollout batch.</li> </ol> <p>The is a primary reference for this style of language-model training pipeline.</p> <h2 id="4-sft-and-reward-model-training">4. SFT and Reward Model Training</h2> <h3 id="supervised-fine-tuning-sft">Supervised Fine-Tuning (SFT)</h3> <p>SFT trains the pretrained model on demonstration responses. The resulting checkpoint initializes the actor and the frozen reference policy in this setup.</p> <p> <figure id="figure-figure-2-supervised-fine-tuning-before-rl-training"> <div class="flex justify-center "> <div class="w-full" ></div> </div><figcaption> Figure 2. Supervised fine-tuning before RL training. </figcaption></figure> </p> <h3 id="reward-model">Reward Model</h3> <p>Train the reward model after SFT and before the PPO stage. Its input is a prompt–response pair, and its output is a scalar preference score.</p> <p> <figure id="figure-figure-3-training-the-reward-model-from-preference-pairs"> <div class="flex justify-center "> <div class="w-full" ></div> </div><figcaption> Figure 3. Training the reward model from preference pairs. </figcaption></figure> </p> <h4 id="initialization">Initialization</h4> <p>Initialize the reward model from the SFT checkpoint. Add a scalar <strong>reward head</strong>, usually a linear projection, to produce the response score:</p> $$ r (x,y) = \text{Linear}(h_\text{final token}) $$<h4 id="preference-loss">Preference Loss</h4> <p>Each preference example contains:</p> <ul> <li>$x$: the prompt</li> <li>$y_w$: the <strong>chosen (preferred)</strong> response</li> <li>$y_l$: the <strong>rejected (less preferred)</strong> response Train the model to assign a higher score to the preferred response. The pairwise loss is:</li> </ul> $$ \mathcal{L}_\text{RM}=-\mathbb{E}_{(x,y_w,y_l)\sim D}[\log\left(\sigma\left(r(x,y_w)-r(x,y_l)\right)\right)] $$<p>where $\sigma(x) = \frac{1}{1 + \exp(-x)}$ is a sigmoid function, hence the loss can be written as:</p> $$ \mathcal{L}_\text{RM}=-\mathbb{E}_{(x,y_w,y_l)\sim D}\left[\log\left(\frac{\exp(r(x,y_w))}{\exp(r(x,y_w))+\exp(r(x,y_l))}\right)\right] $$<p>The following PyTorch class computes the pairwise loss, with an optional margin:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">torch.nn</span> <span class="k">as</span> <span class="nn">nn</span> </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">torch.nn.functional</span> <span class="k">as</span> <span class="nn">F</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">class</span> <span class="nc">PairWiseLoss</span><span class="p">(</span><span class="n">nn</span><span class="o">.</span><span class="n">Module</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="s2">"""Pairwise preference loss for a reward model."""</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"> <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">chosen_reward</span><span class="p">,</span> <span class="n">reject_reward</span><span class="p">,</span> <span class="n">margin</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="n">reward_difference</span> <span class="o">=</span> <span class="n">chosen_reward</span> <span class="o">-</span> <span class="n">reject_reward</span> </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="n">margin</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span> </span></span><span class="line"><span class="cl"> <span class="n">reward_difference</span> <span class="o">=</span> <span class="n">reward_difference</span> <span class="o">-</span> <span class="n">margin</span> </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="o">-</span><span class="n">F</span><span class="o">.</span><span class="n">logsigmoid</span><span class="p">(</span><span class="n">reward_difference</span><span class="p">)</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span> </span></span></code></pre></div><h4 id="interpreting-the-reward-output">Interpreting the Reward Output</h4> <p>In this setup, the reward model assigns a <strong>single scalar score $r(x,y)$ to a completed response</strong>. A common implementation applies a scalar head to the final token’s hidden representation. That representation depends on the preceding context, so the score can summarize the response rather than only its last token.</p> <p>Other aggregation choices include sums, means, weighted sums, or attention-based pooling. The aggregation is an implementation choice; it should match how the reward model was trained.</p> <h2 id="5-ppo-training">5. PPO Training</h2> <h3 id="actor-model">Actor Model</h3> <p>The actor generates responses token by token and is the model being optimized during RL. It begins from the SFT checkpoint; the reward model is already trained and frozen.</p> <p> <figure id="figure-figure-4-updating-the-actor-with-ppo"> <div class="flex justify-center "> <div class="w-full" ></div> </div><figcaption> Figure 4. Updating the actor with PPO. </figcaption></figure> </p> <p>For each rollout batch:</p> <ol> <li><strong>Generate responses.</strong> Sample token IDs from the actor’s output distribution and save their rollout log-probabilities.</li> <li><strong>Score the responses.</strong> The reward model supplies response scores; the reference model supplies token log-probabilities for the KL penalty.</li> <li><strong>Estimate values and advantages.</strong> The critic predicts values for prefixes, which combine with the rewards to produce GAE estimates.</li> <li><strong>Optimize the actor.</strong> Recompute the current policy’s log-probabilities and compare them with the saved rollout probabilities.</li> </ol> <p>Let $\pi_{\mathrm{old}}$ be the rollout policy and $\rho_t(\theta)=\pi_\theta(a_t\mid s_t)/\pi_{\mathrm{old}}(a_t\mid s_t)$. The per-token actor loss is</p> $$ \mathcal{L}_{\mathrm{actor},t} =-\min\left( \rho_t(\theta)\hat A_t, \operatorname{clip}(\rho_t(\theta),1-\epsilon,1+\epsilon)\hat A_t \right). $$<p>The negative sign turns the objective from Part 1 into a loss to minimize. In practice, aggregate over valid response tokens and the batch, masking padding and prompt tokens as appropriate. Treat advantages and old log-probabilities as fixed targets during each update.</p> <p>An entropy bonus can also be included. It is separate from the clipping term and from the reference-policy KL penalty.</p> <h3 id="critic-model">Critic Model</h3> <p>The critic predicts expected future return from each prefix. It learns alongside the actor, but uses a value-prediction loss.</p> <p> <figure id="figure-figure-5-training-the-critic-to-estimate-returns"> <div class="flex justify-center "> <div class="w-full" ></div> </div><figcaption> Figure 5. Training the critic to estimate returns. </figcaption></figure> </p> <p><strong>Initialization.</strong> Possible designs include an SFT-based model with a value head, initialization from the trained reward model, or a value head sharing a backbone with the actor. Separate actor and critic networks are one design choice, not a requirement of every PPO implementation.</p> <p><strong>TD errors and GAE.</strong> For rollout values $V_{w_{\mathrm{old}}}$, define</p> $$ \delta_t=r_t+\gamma(1-d_t)V_{w_{\mathrm{old}}}(s_{t+1}) -V_{w_{\mathrm{old}}}(s_t). $$<p>Here $d_t$ indicates termination. The discount factor is $\gamma$; $\lambda$ instead controls the weighting of TD errors in GAE. For a rollout ending at $T$,</p> $$ \hat A_t^{\mathrm{GAE}}=\sum_{l=0}^{T-t}(\gamma\lambda)^l\delta_{t+l}, \qquad \hat V_t=\hat A_t^{\mathrm{GAE}}+V_{w_{\mathrm{old}}}(s_t). $$<p>Stop the sum at episode boundaries and handle bootstrapping at nonterminal rollout cutoffs. The critic minimizes</p> $$ \mathcal{L}_{\mathrm{critic},t} =\frac12\left(V_w(s_t)-\operatorname{stopgrad}(\hat V_t)\right)^2. $$<p>Using $\hat V_t$ for the return target avoids confusing it with the immediate reward $r_t$.</p> <h3 id="reference-model-and-token-rewards">Reference Model and Token Rewards</h3> <p>The reference policy is usually a frozen copy of the SFT policy. It supplies a fixed point of comparison while the actor changes. By contrast, the <strong>old policy</strong> is refreshed when collecting new rollouts and supplies PPO’s probability-ratio denominator.</p> <p>For a sampled token, the log-probability ratio is</p> $$ k_t=\log\pi_\theta(a_t\mid s_t)-\log\pi_{\mathrm{ref}}(a_t\mid s_t). $$<p>Its expectation under $a_t\sim\pi_\theta(\cdot\mid s_t)$ is the forward KL divergence. A single sampled value is not the full KL divergence and can be negative.</p> <p>For a response ending at token $T$, one common reward decomposition is</p> $$ r_t= \begin{cases} -\beta k_t, & t\lt T,\\ r(x,y)-\beta k_t, & t=T. \end{cases} $$<p>Every token receives the KL shaping term, while the final token also receives the reward model’s response score. Equivalently,</p> $$ r_t=\mathbb{I}[t=T]\,r(x,y)-\beta k_t. $$<p>The terminal reward is propagated to earlier tokens through the return and advantage estimates. During rollout processing, compute these quantities using the rollout policy and hold the resulting targets fixed for PPO updates.</p> <h3 id="practical-training-considerations">Practical Training Considerations</h3> <p>These are useful implementation topics to revisit once the main pipeline is clear:</p> <ul> <li><strong>Pretraining loss and alignment tax:</strong> monitor whether preference optimization reduces performance on other tasks, and consider how auxiliary training objectives affect that trade-off.</li> <li><strong>KL reward scaling:</strong> choose and monitor the penalty coefficient $\beta$.</li> <li><strong>Reward normalization:</strong> make the treatment of reward scale explicit.</li> <li><strong>Advantage normalization:</strong> distinguish this from normalizing the raw reward.</li> <li><strong>Learning rate:</strong> tune actor and critic optimization with their different objectives in mind.</li> </ul> <p> <figure id="figure-figure-6-pretraining-loss-and-the-alignment-tax-source-paperhttpsaclanthologyorg2024emnlp-main35pdf"> <div class="flex justify-center "> <div class="w-full" > </div> </div><figcaption> Figure 6. Pretraining loss and the alignment tax. . </figcaption></figure> </p> <h3 id="open-challenges">Open Challenges</h3> <p>Two questions remain central: does the policy exploit weaknesses in the reward model (<strong>reward hacking</strong>), and do improvements transfer beyond the training prompts (<strong>generalization</strong>)? A higher training reward alone does not answer either question.</p> <h2 id="6-group-relative-policy-optimization-grpo">6. Group Relative Policy Optimization (GRPO)</h2> <h3 id="motivation">Motivation</h3> <p>The PPO pipeline above requires an actor, a critic, a reward signal, and a reference policy. Running these components can be expensive, and the learned critic adds another estimate to fit and maintain.</p> <p> introduces GRPO as a way to obtain an advantage signal from a group of responses to the same prompt, without a separate learned critic.</p> <h3 id="which-component-changes">Which Component Changes?</h3> <ul> <li><strong>Actor:</strong> still generates responses and receives policy updates.</li> <li><strong>Reward signal:</strong> still evaluates responses. Depending on the task, it can come from a model or a verifiable rule.</li> <li><strong>Reference policy:</strong> remains in the original KL-regularized GRPO objective.</li> <li><strong>Critic:</strong> replaced by a baseline derived from rewards within the sampled group.</li> </ul> <p>The key change is how the advantage is estimated. It does not remove the need to evaluate the generated responses.</p> <h3 id="advantage-function-estimation-in-grpo">Advantage Function Estimation in GRPO</h3> <p>GRPO samples a group of responses $\{o_1,\ldots,o_G\}$ to the same prompt using the rollout policy $\pi_{\mathrm{old}}$.</p> <p> <figure id="figure-figure-7-comparison-of-ppo-and-grpo-source-deepseekmathhttpsarxivorgabs240203300"> <div class="flex justify-center "> <div class="w-full" > </div> </div><figcaption> Figure 7. Comparison of PPO and GRPO. Source: . </figcaption></figure> </p> <p>For outcome-level rewards:</p> <p>Scoring the group produces rewards $\mathbf r=(r_1,\ldots,r_G)$. Normalize them within the group:</p> $$ \hat{A}_{i,t}=\hat{r}_i=\frac{r_i-\mathrm{mean}(\mathbf{r})}{\mathrm{std}(\mathbf{r})} $$<p>Each response receives an advantage based on how its reward compares with the group mean, scaled by the group standard deviation. In this outcome-reward formulation, every token in response $i$ shares that advantage. Implementations must also handle zero-variance groups, for example with a small denominator stabilizer.</p> <p> <figure id="figure-figure-8-advantage-estimation-in-ppo-and-grpo"> <div class="flex justify-center "> <div class="w-full" > </div> </div><figcaption> Figure 8. Advantage estimation in PPO and GRPO. </figcaption></figure> </p> <p>The following worked example illustrates response-level rewards:</p> <p> <figure id="figure-figure-9-outcome-reward-example-comparing-responses-source-comfyaihttpscomfyaiapparticlellm-posttrainingoptimizing-ppo-based-algorithms"> <div class="flex justify-center "> <div class="w-full" > </div> </div><figcaption> Figure 9. Outcome reward example: comparing responses. Source: . </figcaption></figure> </p> <p> <figure id="figure-figure-10-outcome-reward-example-continued-source-comfyaihttpscomfyaiapparticlellm-posttrainingoptimizing-ppo-based-algorithms"> <div class="flex justify-center "> <div class="w-full" > </div> </div><figcaption> Figure 10. Outcome reward example, continued. Source: . </figcaption></figure> </p> <h3 id="a-learning-question-how-can-one-advantage-train-every-token">A Learning Question: How Can One Advantage Train Every Token?</h3> <p>I initially found this confusing: if every token in a response receives the same advantage, how can the model learn which tokens to generate?</p> <p>In the unclipped score-function term, the shared weight multiplies a different log-probability gradient at every position:</p> $$ \hat A_i\nabla_\theta\log\pi_\theta(o_{i,t}\mid q,o_{i,\lt t}). $$<p>The weight is response-level, but the gradient depends on the token and its prefix. Different responses receive different weights, so their token sequences contribute differently to the update. Clipping, KL regularization, and shared model parameters also affect the final parameter change.</p> <p>This provides a learning signal without identifying exactly which token caused the final reward. I find it helpful to distinguish <strong>learning from a response-level outcome</strong> from <strong>having precise token-level credit assignment</strong>. They are different claims.</p> <h3 id="the-grpo-objective-and-loss">The GRPO Objective and Loss</h3> <p>The following loss retains the clipped surrogate, uses group-relative advantages, and adds an explicit KL penalty. It is minimized. First, define</p> $$ \rho_{i,t}(\theta) =\frac{\pi_\theta(o_{i,t}\mid q,o_{i,\lt t})} {\pi_{\mathrm{old}}(o_{i,t}\mid q,o_{i,\lt t})}. $$<p>The per-token loss is</p> $$ \begin{aligned} \ell_{i,t}(\theta) ={}&-\min\left( \rho_{i,t}(\theta)\hat A_{i,t}, \operatorname{clip}(\rho_{i,t}(\theta),1-\epsilon,1+\epsilon)\hat A_{i,t} \right)\\ &+\beta k_{i,t}. \end{aligned} $$<p>With prompts $q\sim D$ and response groups sampled from $\pi_{\mathrm{old}}$, average over tokens and responses:</p> $$ \mathcal{L}_{\mathrm{GRPO}}(\theta) =\mathbb{E}_{q,\{o_i\}_{i=1}^G}\left[ \frac1G\sum_{i=1}^G\frac1{|o_i|}\sum_{t=1}^{|o_i|}\ell_{i,t}(\theta) \right]. $$<p>Here $G$ is the group size, $|o_i|$ is the response length, and $k_{i,t}$ is the KL estimator defined below. This describes the original formulation discussed here; alternative implementations can change the normalization or regularization.</p> <p>This uses a group-normalized advantage in place of the critic-based estimate and an explicit KL term.</p> <p>For the sampled token, define $u_{i,t}=\pi_{\mathrm{ref}}(o_{i,t}\mid q,o_{i,\lt t})/\pi_\theta(o_{i,t}\mid q,o_{i,\lt t})$. The KL estimator in this objective is</p> $$ k_{i,t}=u_{i,t}-\log u_{i,t}-1. $$<p>This expression is nonnegative. Under current-policy sampling and the usual support assumptions, its expectation is $D_{\mathrm{KL}}(\pi_\theta\|\pi_{\mathrm{ref}})$. Rollouts are collected with $\pi_{\mathrm{old}}$, so the sampling distribution matters when describing an estimator as unbiased.</p> <p>The objective shown here has no separate entropy bonus. Group sampling supplies multiple candidate responses, but should not be treated as mathematically equivalent to an entropy bonus.</p> <h3 id="actor-training">Actor Training</h3> <p>These diagrams connect the GRPO actor update with the overall rollout and optimization loop:</p> <p> <figure id="figure-figure-11-updating-the-actor-with-grpo"> <div class="flex justify-center "> <div class="w-full" ></div> </div><figcaption> Figure 11. Updating the actor with GRPO. </figcaption></figure> </p> <p> <figure id="figure-figure-12-overview-of-the-grpo-training-pipeline"> <div class="flex justify-center "> <div class="w-full" ></div> </div><figcaption> Figure 12. Overview of the GRPO training pipeline. </figcaption></figure> </p> <h2 id="7-ppo-and-grpo-at-a-glance">7. PPO and GRPO at a Glance</h2> <p>This comparison refers to the PPO-based RLHF pipeline and the outcome-reward GRPO formulation discussed above.</p> <table> <thead> <tr> <th>Aspect</th> <th>PPO pipeline</th> <th>GRPO formulation</th> </tr> </thead> <tbody> <tr> <td>Advantage estimate</td> <td>Critic-based, commonly using GAE</td> <td>Relative rewards within a prompt’s response group</td> </tr> <tr> <td>Learned critic</td> <td>Used here</td> <td>Not required</td> </tr> <tr> <td>Reward signal</td> <td>Learned or otherwise specified</td> <td>Learned or verifiable</td> </tr> <tr> <td>Policy update</td> <td>Clipped surrogate</td> <td>Clipped surrogate with group-relative advantages</td> </tr> <tr> <td>Reference regularization</td> <td>KL shaping in the reward here</td> <td>Explicit KL term in the objective shown</td> </tr> </tbody> </table> <h2 id="key-takeaways">Key Takeaways</h2> <ul> <li>A prompt and response can be viewed as an RL trajectory.</li> <li>Preference learning provides a reward signal; policy optimization uses it.</li> <li>In PPO-based RLHF, the actor, critic, reward model, and reference policy have distinct roles.</li> <li>The rollout policy and the fixed reference policy are different reference points.</li> <li>GRPO removes the learned critic by comparing rewards within a response group.</li> </ul> <p>Return to for the underlying policy-gradient and PPO derivations.</p> <h2 id="acknowledgements">Acknowledgements</h2> <p>These notes would not have been possible without the many excellent books, lecture notes, blog posts, papers, and open-source resources created by the machine learning community. I am deeply grateful to all of the authors and educators whose work helped me understand reinforcement learning and LLM post-training. Many of the explanations and intuitions presented here were inspired by or built upon these resources, although any mistakes, misunderstandings, or inaccuracies are entirely my own.</p> <p>If you are interested in learning more, I highly encourage you to explore the references below. They provide much deeper and more rigorous treatments than these personal notes and have been invaluable throughout my learning journey.</p> <h2 id="references">References</h2> <h3 id="rl-foundations">RL Foundations</h3> <ul> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> </ul> <h3 id="policy-gradient-papers">Policy Gradient Papers</h3> <ul> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> </ul> <h3 id="rlhf">RLHF</h3> <ul> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> </ul> <h3 id="ppo-and-grpo-explanations">PPO and GRPO Explanations</h3> <ul> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> </ul> </article> <article> <h1>Notes on the Foundations of Reinforcement Learning and LLM Post-Training (Part 1)</h1> <p>Mon, 01 Sep 2025 00:00:00 +0000</p> <details class="print:hidden xl:hidden" open> <summary>Table of Contents</summary> <div class="text-sm"> <nav id="TableOfContents"> <ul> <li><a href="#a-few-words-before-we-begin">A Few Words Before We Begin</a> <ul> <li><a href="#how-to-read-these-notes">How to Read These Notes</a></li> </ul> </li> <li><a href="#1-rl-concepts-and-notation">1. RL Concepts and Notation</a> <ul> <li><a href="#what-problem-does-rl-address">What Problem Does RL Address?</a></li> <li><a href="#the-agentenvironment-loop">The Agent–Environment Loop</a></li> <li><a href="#supervised-learning-vs-reinforcement-learning">Supervised Learning vs. Reinforcement Learning</a></li> <li><a href="#key-concepts-and-notation">Key Concepts and Notation</a></li> <li><a href="#rl-algorithm-categories">RL Algorithm Categories</a></li> </ul> </li> <li><a href="#2-policy-gradients">2. Policy Gradients</a></li> <li><a href="#3-the-policy-gradient-theorem">3. The Policy Gradient Theorem</a> <ul> <li><a href="#from-return-to-a-computable-gradient">From Return to a Computable Gradient</a></li> <li><a href="#reward-to-go-and-action-values">Reward-to-Go and Action Values</a></li> <li><a href="#why-variance-matters">Why Variance Matters</a></li> </ul> </li> <li><a href="#4-reinforce-and-baselines">4. REINFORCE and Baselines</a> <ul> <li><a href="#intuition">Intuition</a></li> <li><a href="#algorithm">Algorithm</a></li> <li><a href="#choosing-the-weight-on-the-policy-gradient">Choosing the Weight on the Policy Gradient</a></li> </ul> </li> <li><a href="#5-actorcritic-methods">5. Actor–Critic Methods</a> <ul> <li><a href="#motivation-and-intuition">Motivation and Intuition</a></li> <li><a href="#from-the-bellman-equation-to-a-td-error">From the Bellman Equation to a TD Error</a></li> <li><a href="#updating-the-actor-and-critic">Updating the Actor and Critic</a></li> <li><a href="#algorithm-and-implementation-notes">Algorithm and Implementation Notes</a></li> </ul> </li> <li><a href="#6-trust-region-policy-optimization-trpo">6. Trust Region Policy Optimization (TRPO)</a> <ul> <li><a href="#why-limit-the-policy-update">Why Limit the Policy Update?</a></li> <li><a href="#the-surrogate-objective-and-constraint">The Surrogate Objective and Constraint</a></li> <li><a href="#why-importance-sampling">Why Importance Sampling?</a></li> </ul> </li> <li><a href="#7-proximal-policy-optimization-ppo">7. Proximal Policy Optimization (PPO)</a> <ul> <li><a href="#the-clipped-surrogate-objective">The Clipped Surrogate Objective</a></li> <li><a href="#generalized-advantage-estimation-gae">Generalized Advantage Estimation (GAE)</a></li> <li><a href="#entropy-bonus">Entropy Bonus</a></li> <li><a href="#training-the-value-function">Training the Value Function</a></li> <li><a href="#the-combined-ppo-objective">The Combined PPO Objective</a></li> </ul> </li> <li><a href="#key-takeaways">Key Takeaways</a></li> <li><a href="#acknowledgements">Acknowledgements</a></li> <li><a href="#references">References</a> <ul> <li><a href="#rl-foundations">RL Foundations</a></li> <li><a href="#policy-gradient-papers">Policy Gradient Papers</a></li> <li><a href="#rlhf">RLHF</a></li> <li><a href="#ppo-and-grpo-explanations">PPO and GRPO Explanations</a></li> </ul> </li> </ul> </nav> </div> </details> <h2 id="a-few-words-before-we-begin">A Few Words Before We Begin</h2> <p>Reinforcement learning (RL) has recently emerged as one of the key techniques behind the post-training of large language models (LLMs) and vision-language models (VLMs). As I began reading more about methods such as PPO, RLHF, and GRPO, I realized that I often understood each idea separately but still struggled to see how they were connected. Some of the mathematical details were also quite difficult for me at first, and I found myself returning to the same concepts many times before they started to make sense.</p> <p>I began writing these notes simply to organize what I was learning in my own words. They start from the basic ideas of reinforcement learning and gradually move toward their use in modern LLM post-training. I am sharing them here in the hope that they may be useful to someone else who is also trying to make sense of these concepts.</p> <p>These are personal learning notes, so they may still contain mistakes, unclear explanations, or missing details. I am still learning as well :) If you notice anything that is incorrect or have suggestions for improving these notes, I would genuinely appreciate hearing from you.</p> <h3 id="how-to-read-these-notes">How to Read These Notes</h3> <p>This post follows the path from basic RL concepts to PPO: <strong>concepts → policy gradients → REINFORCE → actor–critic → TRPO → PPO</strong>. Familiarity with probability, derivatives, and neural networks will help with the equations. You can read the intuition first and return to the derivations later.</p> <p> connects these ideas to language-model training, RLHF, and GRPO.</p> <h2 id="1-rl-concepts-and-notation">1. RL Concepts and Notation</h2> <h3 id="what-problem-does-rl-address">What Problem Does RL Address?</h3> <p>In supervised learning, a model learns to predict outputs from examples. Reinforcement learning focuses on <strong>sequential decisions</strong>: an action changes the environment and affects the rewards available later. The goal is to learn a policy that maximizes cumulative reward through interaction.</p> <h3 id="the-agentenvironment-loop">The Agent–Environment Loop</h3> <p> <figure id="figure-figure-1-the-agentenvironment-interaction-loop"> <div class="flex justify-center "> <div class="w-full" > </div> </div><figcaption> Figure 1. The agent–environment interaction loop. </figcaption></figure> </p> <p>The agent interacts with an <strong>environment</strong>. At step $t$, it is in state $s_t \in \mathcal{S}$ and selects an action $a_t \in \mathcal{A}$. The environment produces a reward $r_t$ and a new state, according to the transition distribution $P(s_{t+1}\mid s_t,a_t)$. This transition model may be known or unknown to the agent.</p> <p>One interaction cycle has three steps:</p> <ol> <li><strong>Observe:</strong> receive the current state or observation.</li> <li><strong>Act:</strong> select an action according to the policy.</li> <li><strong>Receive feedback:</strong> observe the reward and the next state.</li> </ol> <p>The cycle continues until the episode ends or the interaction is stopped. The agent’s objective is to improve its decisions based on the return across these interactions.</p> <h3 id="supervised-learning-vs-reinforcement-learning">Supervised Learning vs. Reinforcement Learning</h3> <p>A supervised learning objective typically averages a prediction loss over a fixed dataset.</p> <p> <figure id="figure-figure-2-the-supervised-learning-objective"> <div class="flex justify-center "> <div class="w-full" > </div> </div><figcaption> Figure 2. The supervised learning objective. </figcaption></figure> </p> <p>An RL objective averages return over trajectories generated by a policy. Changing the policy can also change which states and actions appear in the data.</p> <p> <figure id="figure-figure-3-the-reinforcement-learning-objective"> <div class="flex justify-center "> <div class="w-full" > </div> </div><figcaption> Figure 3. The reinforcement learning objective. </figcaption></figure> </p> <table> <thead> <tr> <th>Aspect</th> <th>Supervised learning</th> <th>Reinforcement learning</th> </tr> </thead> <tbody> <tr> <td>Objective</td> <td>Minimize prediction loss</td> <td>Maximize expected return</td> </tr> <tr> <td>Feedback</td> <td>Targets or labels</td> <td>Rewards, which may be delayed</td> </tr> <tr> <td>Data</td> <td>Often a fixed dataset</td> <td>Often collected through interaction</td> </tr> <tr> <td>Decisions</td> <td>Predict an output</td> <td>Choose actions with future consequences</td> </tr> </tbody> </table> <p>These describe common settings; offline RL, for example, also learns from a fixed dataset.</p> <h3 id="key-concepts-and-notation">Key Concepts and Notation</h3> <h4 id="states--observations">States & Observations</h4> <ul> <li><strong>State ($s$):</strong> Information sufficient to describe the environment for predicting the next transition and reward, given an action.</li> <li><strong>Observation ($o$):</strong> The information available to the agent, which may reveal all or only part of the state.</li> </ul> <p><strong>Fully Observed vs. Partially Observed</strong>: If the agent sees the full state $s$, the environment is <strong>fully observed</strong>. If it only sees an observation $o$, it’s <strong>partially observed</strong>.</p> <ul> <li><strong>Representation</strong>: In deep RL, states and observations are typically represented as real-valued vectors, matrices, or tensors.</li> <li><strong>Minor Notes on Notation</strong>: The symbol for state, $s$, is often used in formulas (e.g., $\pi(a|s)$) even when the agent practically only has access to an observation, $o$.</li> </ul> <h4 id="action-spaces">Action Spaces</h4> <ul> <li><strong>Action Space ($\mathcal{A}$)</strong>: The set of all valid actions an agent can take in an environment.</li> <li><strong>Discrete Action Space:</strong> A finite or countable set of actions, such as the legal moves in chess or Go.</li> <li><strong>Continuous Action Space</strong>: Actions are represented by real-valued vectors. For example, the amount of torque to apply to a robot’s motors.</li> </ul> <h4 id="policy">Policy</h4> <ul> <li><strong>Policy ($\pi$)</strong>: A rule the agent uses to decide which action to take. It’s the agent’s “brain.”</li> </ul> <p><strong>Parameterized Policy</strong>: A policy whose behavior is defined by a function with adjustable parameters, $\theta$ (like the weights of a neural network). We can change the agent’s behavior by optimizing these parameters. <strong>Notation</strong>: $\pi_{\theta}(a|s)$ or $\mu_{\theta}(s)$.</p> <ul> <li><strong>Deterministic Policy</strong>: The policy maps a state directly to a single action. <strong>Notation</strong>: $a_t = \mu_{\theta}(s_t)$</li> <li><strong>Stochastic Policy</strong>: The policy outputs a probability distribution over actions.</li> <li><strong>Notation</strong>: $a_t \sim \pi_{\theta}(\cdot | s_t)$</li> <li><strong>Categorical Policy</strong>: Used for <strong>discrete</strong> action spaces. It’s like a classifier that outputs the probability for each possible action.</li> <li><strong>Log-Likelihood</strong>: $\log \pi_{\theta}(a|s) = \log \left[P_{\theta}(s)\right]_a$, where $P_{\theta}(s)$ is the vector of action probabilities.</li> </ul> <p><strong>Diagonal Gaussian Policy</strong>: Used for <strong>continuous</strong> action spaces. The policy outputs a mean $\mu_{\theta}(s)$ and a standard deviation $\sigma_{\theta}(s)$ for a Gaussian distribution.</p> <p><strong>Sampling an Action</strong>: $a = \mu_{\theta}(s) + \sigma_{\theta}(s) \odot z$, where $z \sim \mathcal{N}(0, I)$ is a vector of standard normal noise and $\odot$ is an element-wise product.</p> <ul> <li><strong>Log-Likelihood</strong>: For a $k$-dimensional action $a$:</li> </ul> $$ \log\pi_\theta(a\mid s) = -\frac{1}{2}\left[ \sum_{i=1}^{k}\left(\frac{(a_i-\mu_i)^2}{\sigma_i^2}+2\log\sigma_i\right) +k\log(2\pi)\right]. $$<h4 id="trajectories">Trajectories</h4> <ul> <li>A sequence of states and actions in the world:</li> </ul> $$ \tau = (s_0, a_0, s_1, a_1, ...) $$<ul> <li>The first state $s_0$ is sampled from the start-state distribution, sometimes denoted by $p_0$.</li> <li><strong>State transitions</strong> are determined by the environment.</li> <li><strong>Deterministic</strong>: $s_{t+1} = f(s_t, a_t)$</li> <li><strong>Stochastic</strong>: $s_{t+1} \sim P(\cdot|s_t, a_t)$</li> <li>Trajectories are also frequently called <strong>episodes</strong> or <strong>rollouts</strong>.</li> </ul> <h4 id="rewards-and-return">Rewards and Return</h4> <ul> <li><strong>Reward ($r_t$)</strong>: A scalar feedback signal indicating how well the agent is doing at a given step.</li> <li><strong>Reward Function ($R$)</strong>: Defines the reward, e.g., $r_t = R(s_t, a_t, s_{t+1})$.</li> <li><strong>Return ($R(\tau)$)</strong>: The cumulative reward over a trajectory.</li> <li><strong>Finite-Horizon Undiscounted Return</strong>: The simple sum of rewards over a fixed number of $T$ steps.</li> </ul> <p><strong>Infinite-Horizon Discounted Return</strong>: A sum of all future rewards, discounted by a factor $\gamma \in (0,1)$. This makes rewards in the distant future less valuable than immediate rewards and helps ensure the sum converges.</p> <h4 id="rl-problem-formulation">RL Problem Formulation</h4> <ul> <li>Objective: To find a policy $\pi$ that maximizes the <strong>expected return</strong>.</li> <li><strong>Expected Return ($J(\pi)$)</strong>: The expected return when the agent follows policy $\pi$. This requires averaging over all possible trajectories that could be sampled under $\pi$.</li> <li><strong>Optimization Problem</strong>: Find the optimal policy, $\pi^*$, that yields the maximum possible expected return.</li> </ul> <h4 id="value-function">Value Function</h4> <p>Value functions estimate the expected return from a given state or state-action pair.</p> <p><strong>On-Policy Value Function ($V^{\pi}(s)$)</strong>: The expected return if you start in state $s$ and follow policy $\pi$ forever. $V^{\pi}(s) = \underset{\tau \sim \pi}{\mathbb{E}} \left[ R(\tau) \mid s_0 = s \right]$</p> <p><strong>On-Policy Action-Value Function ($Q^{\pi}(s,a)$)</strong>: The expected return if you start in state $s$, take action $a$, and <em>then</em> follow policy $\pi$ forever. $Q^{\pi}(s, a) = \underset{\tau \sim \pi}{\mathbb{E}} \left[ R(\tau) \mid s_0 = s, a_0 = a \right]$</p> <ul> <li><strong>Optimal Value Function ($V^*(s)$)</strong>: The maximum possible expected return from state $s$, achieved by following the optimal policy $V^{*}(s) = \max_{\pi} V^{\pi}(s)$</li> </ul> <p><strong>Optimal Action-Value Function ($Q^*(s,a)$)</strong>: The maximum expected return starting from $s$, taking action $a$, and <em>then</em> following the optimal policy forever. $Q^{*}(s, a) = \max_{\pi} Q^{\pi}(s, a)$</p> <ul> <li><strong>Optimal Action</strong>: If you know $Q^*(s,a)$, you can find the optimal action by choosing the one with the highest value $a^{*}_s = \arg \max_{a}Q^{*}(s, a)$</li> <li><strong>Bellman Equations</strong>: These are fundamental self-consistency equations. The value of a state is the immediate reward plus the discounted value of the next state.</li> </ul> <p><strong>On-Policy</strong>:</p> $$ V^{\pi}(s) = \underset{a \sim \pi, s'\sim P}{\mathbb{E}}[r(s,a) + \gamma V^{\pi}(s')] $$$$ Q^{\pi}(s,a) = \underset{s'\sim P}{\mathbb{E}}[r(s,a) + \gamma \underset{a'\sim \pi}{\mathbb{E}}[Q^{\pi}(s',a')]] $$<p><strong>Optimal</strong>:</p> $$ V^*(s) = \max_a \underset{s'\sim P}{\mathbb{E}}[r(s,a) + \gamma V^*(s')] $$$$ Q^*(s,a) = \underset{s'\sim P}{\mathbb{E}}[r(s,a) + \gamma \max_{a'} Q^*(s',a')] $$<p><strong>Advantage Function ($A^{\pi}(s,a)$)</strong>: Measures how much better taking action $a$ is compared to the average action from policy $\pi$ in state $s$. It is crucial for many policy gradient algorithms</p> $$ A^{\pi}(s, a) = Q^{\pi}(s, a) - V^{\pi}(s) $$<h4 id="notation-summary">Notation Summary</h4> <p>Throughout these notes, $r_t$ is the reward received after action $a_t$, and $G_t$ is the return from that step onward. Some references instead call that reward $r_{t+1}$; the indexing convention differs, not the underlying idea.</p> $$ G_t = \sum_{k=0}^{\infty}\gamma^k r_{t+k} = r_t + \gamma G_{t+1}. $$<table> <thead> <tr> <th>Symbol</th> <th>Meaning</th> </tr> </thead> <tbody> <tr> <td>$s_t$, $a_t$, $r_t$</td> <td>State, action, and immediate reward at step $t$</td> </tr> <tr> <td>$\pi_\theta(a_t \mid s_t)$</td> <td>Policy parameterized by $\theta$</td> </tr> <tr> <td>$G_t$</td> <td>Discounted return from step $t$</td> </tr> <tr> <td>$V^\pi(s)$</td> <td>Expected return from state $s$ under policy $\pi$</td> </tr> <tr> <td>$Q^\pi(s,a)$</td> <td>Expected return after action $a$ in state $s$, then following $\pi$</td> </tr> <tr> <td>$A^\pi(s,a)$</td> <td>Advantage: $Q^\pi(s,a)-V^\pi(s)$</td> </tr> <tr> <td>$V^*(s)$, $Q^*(s,a)$</td> <td>Optimal state and action values</td> </tr> <tr> <td>$\gamma$</td> <td>Discount factor</td> </tr> </tbody> </table> <p>The value functions are related by</p> $$ V^\pi(s)=\mathbb{E}_{a\sim\pi(\cdot\mid s)}[Q^\pi(s,a)]. $$<h3 id="rl-algorithm-categories">RL Algorithm Categories</h3> <ul> <li>Based on how to update: <strong>On-policy vs. Off-policy Learning</strong></li> </ul> <p><strong>On-policy</strong>: The behavior policy (used to collect samples) and the target policy (used to update) are the <strong>same policy</strong>. For example, in SARSA, the update uses a five-tuple sampled using an $\epsilon$-greedy policy: $(s,a,r,s',a')$</p> <p><strong>Off-Policy:</strong> The behavior policy (used to collect samples) and the target policy (used to update) are <strong>not the same</strong>. For example, in Q-learning, the update uses four-tuples data $(s,a,r,s')$ and $a'$ is obtained via $a^*=\arg\max_{a\prime}Q(s\prime,a\prime)$, rather than sampled from the behavior policy.</p> <ul> <li>Based on the access to environment model: <strong>Model-based vs. Model-free</strong></li> </ul> <p>An environment model describes how the environment responds to actions.</p> <ul> <li><strong>Model-based</strong>: Rely on the model of the environment; either the model is known or the algorithm learns it explicitly.</li> <li><strong>Model-free</strong>: No dependency on the model during learning.</li> <li>Based on how the policy is learned:</li> <li><strong>Value-based:</strong> Learn a value function and derive a policy from it, rather than learning a separately parameterized policy.</li> <li><strong>Policy-based:</strong> Directly optimize a parameterized policy.</li> </ul> <h2 id="2-policy-gradients">2. Policy Gradients</h2> <p>We want to optimize the parameters $\theta$ of a stochastic policy $\pi_\theta$. Write the objective as expected return over trajectories:</p> $$ J(\theta)=\mathbb{E}_{\tau\sim\pi_\theta}[R(\tau)]. $$<p>A trajectory begins at a state drawn from the initial-state distribution $p_0$. Equivalently, $J(\theta)=\mathbb{E}_{s_0\sim p_0}[V^{\pi_\theta}(s_0)]$. The initial-state distribution should not be confused with the distribution of states visited later under the policy.</p> <p>Using <strong>gradient ascent</strong>, we update the policy in a direction that increases this objective:</p> $$ \theta\leftarrow\theta+\alpha\nabla_\theta J(\theta). $$<p>The challenge is finding a gradient estimator from sampled interactions, even when the environment’s transition model is unknown.</p> <h2 id="3-the-policy-gradient-theorem">3. The Policy Gradient Theorem</h2> <h3 id="from-return-to-a-computable-gradient">From Return to a Computable Gradient</h3> <p>The policy affects both the actions chosen and the states visited. The policy gradient theorem lets us express the gradient without explicitly differentiating the state-visitation distribution.</p> <p>For the finite-horizon, <strong>undiscounted</strong> case in this derivation, let $R(\tau)=\sum_{t=0}^{T-1}r_t$. The log-derivative identity is</p> $$ \nabla_\theta p_\theta(\tau) =p_\theta(\tau)\nabla_\theta\log p_\theta(\tau). $$<p>If the environment dynamics do not depend on $\theta$, only the policy terms contribute to the trajectory log-probability gradient:</p> $$ \nabla_\theta\log p_\theta(\tau) =\sum_{t=0}^{T-1}\nabla_\theta\log\pi_\theta(a_t\mid s_t). $$<p>Combining these identities gives</p> $$ \begin{aligned} \nabla_\theta J(\theta) &=\nabla_\theta\int p_\theta(\tau)R(\tau)\,d\tau\\ &=\mathbb{E}_{\tau\sim\pi_\theta}\left[ R(\tau)\sum_{t=0}^{T-1}\nabla_\theta\log\pi_\theta(a_t\mid s_t) \right]. \end{aligned} $$<p>The useful pattern is <strong>a return signal multiplied by the gradient of the action’s log-probability</strong>. We can estimate the expectation with sampled trajectories. See for the full argument.</p> <h3 id="reward-to-go-and-action-values">Reward-to-Go and Action Values</h3> <p>Rewards received before an action do not help evaluate that action. We can instead use its reward-to-go, $G_t=\sum_{k=t}^{T-1}r_k$ in this undiscounted derivation:</p> $$ \nabla_\theta J(\theta) =\mathbb{E}_{\tau\sim\pi_\theta}\left[ \sum_{t=0}^{T-1}G_t\nabla_\theta\log\pi_\theta(a_t\mid s_t) \right]. $$<p>Conditioning on $(s_t,a_t)$ replaces $G_t$ with its expectation, $Q^\pi(s_t,a_t)$. This connects the trajectory estimator to the action-value form of the policy gradient. In a discounted objective, the corresponding time weights must also be included consistently.</p> <h3 id="why-variance-matters">Why Variance Matters</h3> <p><strong>Bias</strong> is systematic error: an unbiased estimator equals the target gradient on average. <strong>Variance</strong> describes how much the estimate changes across sampled batches.</p> <p>Monte Carlo policy-gradient estimates can be unbiased while still having high variance. Different trajectories can produce very different returns, so a small batch may give a noisy update. The next sections introduce baselines and learned value functions to make that signal more useful.</p> <h2 id="4-reinforce-and-baselines">4. REINFORCE and Baselines</h2> <h3 id="intuition">Intuition</h3> <p>REINFORCE estimates the policy gradient using <strong>complete episodes</strong>. For each action, it uses the observed return from that point onward as a Monte Carlo estimate of the action value.</p> $$ \mathbb{E}[G_t\mid s_t,a_t]=Q^\pi(s_t,a_t). $$<p>This avoids learning a critic, but the return depends on all the later transitions and rewards. That is the source of its noisy updates.</p> <h3 id="algorithm">Algorithm</h3> <p>Choose a learning rate $\alpha$, discount factor $\gamma$, and number of episodes. Initialize the policy parameters $\theta$.</p> <ol> <li><strong>Collect a complete episode.</strong> Sample actions from the policy and store states, actions, and rewards.</li> <li><strong>Compute returns backward.</strong> Start with $G=0$ and apply $G\leftarrow r_t+\gamma G$ from the final step to the first.</li> <li><strong>Accumulate the policy gradient.</strong> Weight each action’s log-probability gradient by its return.</li> <li><strong>Update the policy.</strong> Take a gradient-ascent step, then collect fresh data.</li> </ol> <p>For the discounted start-state objective, an episode estimate is</p> $$ \hat g=\sum_{t=0}^{T-1}\gamma^tG_t\nabla_\theta\log\pi_\theta(a_t\mid s_t), \qquad \theta\leftarrow\theta+\alpha\hat g. $$<p>Setting $\gamma=1$ recovers the undiscounted estimator above. The following is <strong>pseudocode</strong>, with environment and policy operations left abstract:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">initialize policy parameters theta </span></span><span class="line"><span class="cl">repeat for each episode: </span></span><span class="line"><span class="cl"> collect states, actions, rewards using the current policy </span></span><span class="line"><span class="cl"> G = 0 </span></span><span class="line"><span class="cl"> returns = an array with one entry per action </span></span><span class="line"><span class="cl"> for t from T - 1 down to 0: </span></span><span class="line"><span class="cl"> G = rewards[t] + gamma * G </span></span><span class="line"><span class="cl"> returns[t] = G </span></span><span class="line"><span class="cl"> gradient = 0 </span></span><span class="line"><span class="cl"> for t from 0 to T - 1: </span></span><span class="line"><span class="cl"> gradient += gamma**t * returns[t] * grad_log_policy(actions[t], states[t]) </span></span><span class="line"><span class="cl"> theta += learning_rate * gradient </span></span></code></pre></div><h3 id="choosing-the-weight-on-the-policy-gradient">Choosing the Weight on the Policy Gradient</h3> <p>The general pattern is a score-function gradient multiplied by a weight $\psi_t$. Several choices connect REINFORCE to actor–critic methods:</p> <ul> <li><strong>Whole-trajectory return:</strong> every action receives the same trajectory-level signal.</li> <li><strong>Reward-to-go $G_t$:</strong> each action receives only the rewards from that step onward.</li> <li><strong>Baseline-adjusted return $G_t-b(s_t)$:</strong> subtract a state-dependent baseline to reduce variance without changing the expected policy gradient.</li> <li><strong>Action value $Q^\pi(s_t,a_t)$:</strong> use the expected return conditioned on the state and action.</li> <li><strong>Advantage $A^\pi(s_t,a_t)$:</strong> subtract $V^\pi(s_t)$ from the action value to compare an action with the policy’s average behavior in that state.</li> <li><strong>TD-based advantage estimate:</strong> use a learned critic and observed transitions, as described next.</li> </ul> <p>A useful baseline should explain variation in return without depending on the sampled action. Its purpose is to improve the gradient estimate, rather than change which policy we want to learn.</p> <h2 id="5-actorcritic-methods">5. Actor–Critic Methods</h2> <h3 id="motivation-and-intuition">Motivation and Intuition</h3> <p>An <strong>actor</strong> learns the policy, while a <strong>critic</strong> learns a value function that helps evaluate the actor’s decisions. Instead of waiting for a complete Monte Carlo return, we can combine an observed reward with the critic’s prediction of what happens next.</p> <p>The actor has parameters $\theta$ and policy $\pi_\theta(a\mid s)$. Here the critic has parameters $w$ and predicts $V_w(s)$. Other actor–critic algorithms may learn an action-value function instead.</p> <h3 id="from-the-bellman-equation-to-a-td-error">From the Bellman Equation to a TD Error</h3> <p>Recall the relationship between action values and state values:</p> $$ Q^\pi(s,a)=\mathbb{E}_{r,s'\mid s,a}[r+\gamma V^\pi(s')]. $$<p>For a sampled transition, the one-step target and temporal-difference (TD) error are</p> $$ y_t=r_t+\gamma(1-d_t)V_w(s_{t+1}), \qquad \delta_t=y_t-V_w(s_t), $$<p>where $d_t=1$ at a terminal state and $0$ otherwise. The terminal mask prevents bootstrapping beyond the end of an episode.</p> <p>With an exact value function, the conditional expectation of this TD error is the advantage. A sampled transition remains noisy, and an imperfect learned value function can introduce bias. Stochastic transitions alone do not make the conditional estimate biased.</p> <h3 id="updating-the-actor-and-critic">Updating the Actor and Critic</h3> <p>The actor uses $\delta_t$ as an advantage estimate. A per-transition update direction is</p> $$ \hat g_t=\delta_t\nabla_\theta\log\pi_\theta(a_t\mid s_t). $$<p>A positive TD error encourages the sampled action; a negative one discourages it. The critic minimizes a squared prediction error:</p> $$ \mathcal{L}_V(w)=\frac12\left(V_w(s_t)-\operatorname{stopgrad}(y_t)\right)^2. $$<p>Holding the target fixed gives</p> $$ \nabla_w\mathcal{L}_V(w)=-\delta_t\nabla_wV_w(s_t). $$<p>Consequently, gradient descent on this loss updates $w$ in the direction $+\delta_t\nabla_wV_w(s_t)$. Keeping the loss gradient and the parameter-update direction separate makes the sign easier to follow.</p> <h3 id="algorithm-and-implementation-notes">Algorithm and Implementation Notes</h3> <ol> <li><strong>Act:</strong> sample $a_t\sim\pi_\theta(\cdot\mid s_t)$.</li> <li><strong>Observe:</strong> receive $r_t$, $s_{t+1}$, and the terminal flag.</li> <li><strong>Evaluate:</strong> compute the target $y_t$ and TD error $\delta_t$.</li> <li><strong>Update the critic:</strong> minimize the value prediction loss with the target held fixed.</li> <li><strong>Update the actor:</strong> use the detached TD error to weight the log-probability gradient.</li> </ol> <p>In PyTorch, <code>.detach()</code> stops gradients through a tensor. For the critic, detach the bootstrap target; for the actor, detach the advantage estimate so that the actor loss does not also optimize the critic through that weight.</p> <p>Implementations often collect a small batch of transitions before updating. Batch size and rollout length affect the trade-off between noisy estimates and delayed updates.</p> <h2 id="6-trust-region-policy-optimization-trpo">6. Trust Region Policy Optimization (TRPO)</h2> <h3 id="why-limit-the-policy-update">Why Limit the Policy Update?</h3> <p>This section provides background for PPO rather than a full TRPO derivation. A large parameter update can change a neural-network policy substantially and reduce its performance. TRPO uses a <strong>trust region</strong> to limit the change while improving an advantage-based surrogate objective.</p> <p>We collect data with an old policy $\pi_{\mathrm{old}}$, then evaluate candidate policies against the old policy’s advantages. Keeping that reference fixed is essential: the expected advantage of a policy under its own action distribution is zero.</p> <h3 id="the-surrogate-objective-and-constraint">The Surrogate Objective and Constraint</h3> <p>Define $d_{\mathrm{old}}$ as the state-visitation distribution used by the surrogate and $\rho_\theta(s,a)=\pi_\theta(a\mid s)/\pi_{\mathrm{old}}(a\mid s)$. The practical TRPO optimization problem combines an advantage-weighted objective with an average KL constraint:</p> $$ \begin{aligned} \max_\theta\quad & \mathbb{E}_{s\sim d_{\mathrm{old}},\,a\sim\pi_{\mathrm{old}}} \left[\rho_\theta(s,a)A^{\pi_{\mathrm{old}}}(s,a)\right]\\ \text{subject to}\quad & \mathbb{E}_{s\sim d_{\mathrm{old}}}\left[ D_{\mathrm{KL}}\left(\pi_{\mathrm{old}}(\cdot\mid s)\,\|\,\pi_\theta(\cdot\mid s)\right) \right]\leq\delta. \end{aligned} $$<p>The <strong>importance weight</strong> adjusts an action’s contribution according to how likely the candidate policy is to choose it relative to the rollout policy. For a positive advantage, increasing the ratio increases its contribution to the surrogate; decreasing the ratio reduces that contribution. Negative advantages reverse the preference.</p> <p>The <strong>KL constraint</strong> limits the average distributional change over the sampled states. The threshold $\delta$ controls the size of the trust region. This helps keep the local surrogate useful as the policy changes; it is not a guarantee that every individual state or action changes by a fixed amount.</p> <p>See and the for the optimization procedure.</p> <h3 id="why-importance-sampling">Why Importance Sampling?</h3> <p>Suppose we want an expectation under a target distribution $p$, but our samples come from $q$. If $q(x)>0$ wherever $p(x)>0$, we can rewrite</p> $$ \begin{aligned} \mathbb{E}_{x\sim p}[h(x)] &=\int h(x)p(x)\,dx\\ &=\int h(x)\frac{p(x)}{q(x)}q(x)\,dx\\ &=\mathbb{E}_{x\sim q}\left[\frac{p(x)}{q(x)}h(x)\right]. \end{aligned} $$<p>In the policy surrogate, the ratio reweights actions from the old policy. The surrogate still uses the old state distribution, so it should not be mistaken for an exact evaluation of the candidate policy’s full return.</p> <p>This is also useful background for PPO: collect a rollout batch, save its action log-probabilities, and optimize a surrogate over several passes through that batch. As the policy moves away from the rollout policy, the approximation becomes less reliable. Policy-change controls help address that problem.</p> <h2 id="7-proximal-policy-optimization-ppo">7. Proximal Policy Optimization (PPO)</h2> <h3 id="the-clipped-surrogate-objective">The Clipped Surrogate Objective</h3> <p>TRPO solves a constrained optimization problem. PPO-Clip uses a simpler clipped surrogate objective that can be optimized with ordinary gradient-based updates.</p> <p>Define the new-to-old probability ratio as</p> $$ \rho_t(\theta)=\frac{\pi_\theta(a_t\mid s_t)}{\pi_{\mathrm{old}}(a_t\mid s_t)}. $$<p>The old policy is the policy that collected the rollout. Let $\hat A_t$ be an advantage estimate computed from that rollout. PPO maximizes</p> $$ L^{\mathrm{CLIP}}(\theta)=\mathbb{E}_t\left[ \min\left(\rho_t(\theta)\hat A_t, \operatorname{clip}(\rho_t(\theta),1-\epsilon,1+\epsilon)\hat A_t\right) \right]. $$<p>Clipping removes the incentive for certain changes that move the probability ratio too far in a favorable direction. It <strong>does not enforce a hard bound</strong> on the actual ratio or guarantee a small KL divergence. See .</p> <p>Here I use $\rho_t$ for the probability ratio so it is visually distinct from the reward $r_t$.</p> <h3 id="generalized-advantage-estimation-gae">Generalized Advantage Estimation (GAE)</h3> <p>The advantage tells us how much better an action is than the policy’s average behavior in that state. Two ways to estimate it are:</p> <ul> <li><strong>Monte Carlo:</strong> use the full observed return minus a baseline. This avoids bootstrapping at an episode’s end, but may have high variance.</li> <li><strong>One-step TD:</strong> combine an immediate reward with the critic’s next-state estimate. This often reduces variance but depends on the accuracy of the critic.</li> </ul> <p>GAE interpolates between these approaches. For readability, the equations below omit terminal masks; at episode boundaries, stop bootstrapping and truncate the sums appropriately.</p> <p>Start with the TD error $\delta_t=r_t+\gamma V(s_{t+1})-V(s_t)$. Multi-step estimates are</p> $$ \begin{aligned} \hat A_t^{(1)}&=\delta_t,\\ \hat A_t^{(2)}&=\delta_t+\gamma\delta_{t+1},\\ \hat A_t^{(3)}&=\delta_t+\gamma\delta_{t+1}+\gamma^2\delta_{t+2},\\ \hat A_t^{(k)}&=\sum_{i=0}^{k-1}\gamma^i\delta_{t+i}. \end{aligned} $$<p>Expanding the TD errors makes the intermediate value terms cancel:</p> $$ \hat A_t^{(k)} =-V(s_t)+\sum_{i=0}^{k-1}\gamma^i r_{t+i}+\gamma^kV(s_{t+k}). $$<p>For $0\leq\lambda\lt 1$, an exponentially weighted average gives</p> $$ \begin{aligned} \hat A_t^{\mathrm{GAE}} &=(1-\lambda)\sum_{k=1}^{\infty}\lambda^{k-1}\hat A_t^{(k)}\\ &=(1-\lambda)\sum_{k=1}^{\infty}\lambda^{k-1} \sum_{l=0}^{k-1}\gamma^l\delta_{t+l}\\ &=\sum_{l=0}^{\infty}\gamma^l\delta_{t+l} \left[(1-\lambda)\sum_{k=l+1}^{\infty}\lambda^{k-1}\right]\\ &=\sum_{l=0}^{\infty}(\gamma\lambda)^l\delta_{t+l}. \end{aligned} $$<p>The parameter $\lambda$ controls the trade-off:</p> <ul> <li>At $\lambda=0$, GAE reduces to the one-step TD error.</li> <li>At $\lambda=1$, the finite-episode sum reduces to the Monte Carlo return minus the baseline when the terminal value is zero.</li> <li>Intermediate values trade reliance on long sampled returns against reliance on learned value estimates.</li> </ul> <p>The develops this bias–variance perspective in more detail.</p> <h3 id="entropy-bonus">Entropy Bonus</h3> <p>An entropy bonus encourages exploration by discouraging the policy from becoming too concentrated too early. Without sufficient exploration, the agent may commit to a suboptimal action before discovering alternatives.</p> <p>Formula:</p> $$ S[\pi_\theta](s_t) = H(\theta) = - \mathbb{E}_{a_t} [\log \pi_\theta (a_t | s_t)] = -\sum_a\pi_\theta (a | s_t) \log \pi_\theta (a | s_t) $$<p>For categorical logits, the following PyTorch snippet computes the entropy. It assumes <code>logits_current</code> contains valid, unpadded positions:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">torch.nn.functional</span> <span class="k">as</span> <span class="nn">F</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Compute entropy of current policy</span> </span></span><span class="line"><span class="cl"><span class="n">probs_current</span> <span class="o">=</span> <span class="n">F</span><span class="o">.</span><span class="n">softmax</span><span class="p">(</span><span class="n">logits_current</span><span class="p">,</span> <span class="n">dim</span><span class="o">=-</span><span class="mi">1</span><span class="p">)</span> </span></span><span class="line"><span class="cl"><span class="n">log_probs_current</span> <span class="o">=</span> <span class="n">F</span><span class="o">.</span><span class="n">log_softmax</span><span class="p">(</span><span class="n">logits_current</span><span class="p">,</span> <span class="n">dim</span><span class="o">=-</span><span class="mi">1</span><span class="p">)</span> </span></span><span class="line"><span class="cl"><span class="n">entropy</span> <span class="o">=</span> <span class="o">-</span><span class="p">(</span><span class="n">probs_current</span> <span class="o">*</span> <span class="n">log_probs_current</span><span class="p">)</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">dim</span><span class="o">=-</span><span class="mi">1</span><span class="p">)</span> </span></span><span class="line"><span class="cl"><span class="c1"># Average over sequence and batch</span> </span></span><span class="line"><span class="cl"><span class="n">entropy_bonus</span> <span class="o">=</span> <span class="n">entropy</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span> </span></span></code></pre></div><h3 id="training-the-value-function">Training the Value Function</h3> <p>PPO also fits a critic. Let $V_w$ be the current value function and let $\hat V_t$ be a return target computed from the rollout. With GAE, a common target is</p> $$ \hat V_t=\hat A_t^{\mathrm{GAE}}+V_{w_{\mathrm{old}}}(s_t). $$<p>This is a bootstrapped return target; it is not generally identical to the observed Monte Carlo return. Fit the critic with the target held fixed:</p> $$ \mathcal{L}^{\mathrm{VF}}_t(w) =\frac12\left(V_w(s_t)-\operatorname{stopgrad}(\hat V_t)\right)^2. $$<p>Average the loss over the batch. The actor and critic may use separate networks or share a backbone with different output heads. Here, $\theta$ and $w$ distinguish the policy and value parameters even when discussing their losses together.</p> <h3 id="the-combined-ppo-objective">The Combined PPO Objective</h3> <p>One common combined objective is <strong>maximized</strong>:</p> $$ J_{\mathrm{PPO}}(\theta,w) =\mathbb{E}_t\left[ L_t^{\mathrm{CLIP}}(\theta) -c_1\mathcal{L}_t^{\mathrm{VF}}(w) +c_2H(\pi_\theta(\cdot\mid s_t)) \right]. $$<p>A loss minimized by an optimizer has the opposite sign. The clipped term updates the actor, the value loss trains the critic, and the entropy term encourages exploration.</p> <table> <thead> <tr> <th>Parameter</th> <th>Role</th> </tr> </thead> <tbody> <tr> <td>$\gamma$</td> <td>Discounting future rewards</td> </tr> <tr> <td>$\lambda$</td> <td>GAE’s bias–variance trade-off</td> </tr> <tr> <td>$\epsilon$</td> <td>Clipping threshold in the surrogate objective</td> </tr> <tr> <td>$c_1$</td> <td>Weight of the value loss</td> </tr> <tr> <td>$c_2$</td> <td>Weight of the entropy bonus</td> </tr> </tbody> </table> <h2 id="key-takeaways">Key Takeaways</h2> <ul> <li>Policy gradients connect sampled rewards to changes in action probabilities.</li> <li>REINFORCE uses complete returns; a baseline can reduce variance.</li> <li>Actor–critic methods learn value estimates to guide policy updates.</li> <li>TRPO constrains policy change, while PPO uses a clipped surrogate.</li> <li>GAE, value targets, and entropy each address a different part of training.</li> </ul> <hr> <p>Continue with , which covers RL in the LLM/VLM setting, RLHF, the full PPO training pipeline, and GRPO.</p> <h2 id="acknowledgements">Acknowledgements</h2> <p>These notes would not have been possible without the many excellent books, lecture notes, blog posts, papers, and open-source resources created by the machine learning community. I am deeply grateful to all of the authors and educators whose work helped me understand reinforcement learning and LLM post-training. Many of the explanations and intuitions presented here were inspired by or built upon these resources, although any mistakes, misunderstandings, or inaccuracies are entirely my own.</p> <p>If you are interested in learning more, I highly encourage you to explore the references below. They provide much deeper and more rigorous treatments than these personal notes and have been invaluable throughout my learning journey.</p> <h2 id="references">References</h2> <h3 id="rl-foundations">RL Foundations</h3> <ul> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> </ul> <h3 id="policy-gradient-papers">Policy Gradient Papers</h3> <ul> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> </ul> <h3 id="rlhf">RLHF</h3> <ul> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> </ul> <h3 id="ppo-and-grpo-explanations">PPO and GRPO Explanations</h3> <ul> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> <li> <p> </p> </li> </ul> </article> <article> <h1>Clinical Features and Survival Analysis of Osteosarcoma Patients: A Vietnamese Nationwide Retrospective Study</h1> <p>Mon, 01 Jan 2024 00:00:00 +0000</p> <p>Retrospective report on osteosarcoma patient characteristics and survival analysis in the Vietnamese context.</p> </article> <article> <h1>Example Talk: Recent Work</h1> <p>Mon, 01 Jan 2024 00:00:00 +0000</p>  <h1 id="example-talk">Example Talk</h1> <h3 id="dr-alex-johnson--meta-ai">Dr. Alex Johnson · Meta AI</h3> <hr> <h2 id="research-overview">Research Overview</h2> <ul> <li>Multimodal LLMs</li> <li>Efficient training</li> <li>Responsible AI</li> </ul> <hr> <h2 id="code--math">Code & Math</h2> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">score</span><span class="p">(</span><span class="n">x</span><span class="p">:</span> <span class="nb">int</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span> </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">x</span> <span class="o">*</span> <span class="n">x</span> </span></span></code></pre></div>$$ E = mc^2 $$<hr> <h2 id="dual-column-layout">Dual Column Layout</h2> <div class="r-hstack"> <div style="flex: 1; padding-right: 1rem;"> <h3 id="left-column">Left Column</h3> <ul> <li>Point A</li> <li>Point B</li> <li>Point C</li> </ul> </div> <div style="flex: 1; padding-left: 1rem;"> <h3 id="right-column">Right Column</h3> <ul> <li>Detail 1</li> <li>Detail 2</li> <li>Detail 3</li> </ul> </div> </div> <hr>  <div style="display: flex; gap: 2rem;"> <div style="flex: 2;"> <h3 id="main-content-23-width">Main Content (2/3 width)</h3> <p>This column takes up twice the space of the right column.</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">example</span><span class="p">():</span> </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="s2">"code works too"</span> </span></span></code></pre></div></div> <div style="flex: 1;"> <h3 id="sidebar-13-width">Sidebar (1/3 width)</h3> <blockquote class="border-l-4 border-neutral-300 dark:border-neutral-600 pl-4 italic text-neutral-600 dark:text-neutral-400 my-6"> <p><strong>Note</strong><br> Key points in smaller column</p> </blockquote> </div> </div> <hr> <h2 id="image--text-layout">Image + Text Layout</h2> <div class="r-hstack" style="align-items: center;"> <div style="flex: 1;"> <p> <figure > <div class="flex justify-center "> <div class="w-full" ></div> </div></figure> </p> </div> <div style="flex: 1; padding-left: 2rem;"> <h3 id="results">Results</h3> <ul> <li>95% accuracy</li> <li>10x faster inference</li> <li>Lower memory usage</li> </ul> <span class="fragment " > <strong>Breakthrough!</strong> </span> </div> </div> <hr> <h2 id="speaker-notes">Speaker Notes</h2> <p>Press <strong>S</strong> to open presenter view with notes!</p> <p>This slide has hidden speaker notes below.</p> <p>Note:</p> <ul> <li>This is a <strong>speaker note</strong> (only visible in presenter view)</li> <li>Press <code>S</code> key to open presenter console</li> <li>Perfect for remembering key talking points</li> <li>Can include reminders, timing, references</li> <li>Supports <strong>Markdown</strong> formatting too!</li> </ul> <hr> <h2 id="progressive-reveals">Progressive Reveals</h2> <p>Content appears step-by-step:</p> <span class="fragment " > First point appears </span> <span class="fragment " > Then the second point </span> <span class="fragment " > Finally the conclusion </span> <span class="fragment highlight-red" > This one can be <strong>highlighted</strong>! </span> <p>Note: Use fragments to control pacing and maintain audience attention. Each fragment appears on click.</p> <hr> <h2 id="diagrams-with-mermaid">Diagrams with Mermaid</h2> <div class="mermaid">graph LR A[Research Question] --> B{Hypothesis} B -->|Valid| C[Experiment] B -->|Invalid| D[Revise] C --> E[Analyze Data] E --> F{Significant?} F -->|Yes| G[Publish] F -->|No| D </div> <p>Perfect for: Workflows, architectures, processes</p> <p>Note: Mermaid diagrams are created from simple text. They’re version-controllable and edit anywhere!</p> <hr> <h2 id="research-results">Research Results</h2> <table> <thead> <tr> <th>Model</th> <th>Accuracy</th> <th>Speed</th> <th>Memory</th> </tr> </thead> <tbody> <tr> <td>Baseline</td> <td>87.3%</td> <td>1.0x</td> <td>2GB</td> </tr> <tr> <td>Ours (v1)</td> <td>92.1%</td> <td>1.5x</td> <td>1.8GB</td> </tr> <tr> <td><strong>Ours (v2)</strong></td> <td><strong>95.8%</strong></td> <td><strong>2.3x</strong></td> <td><strong>1.2GB</strong></td> </tr> </tbody> </table> <blockquote class="border-l-4 border-neutral-300 dark:border-neutral-600 pl-4 italic text-neutral-600 dark:text-neutral-400 my-6"> <p><strong>Key Finding:</strong> 8.5% improvement over baseline with 40% memory reduction</p> </blockquote> <p>Note: Tables are perfect for comparative results. Markdown tables are simple and version-control friendly.</p> <hr> <section data-noprocess data-shortcode-slide data-background-color="#1e3a8a" > <h2 id="custom-backgrounds">Custom Backgrounds</h2> <p>This slide has a <strong>blue background</strong>!</p> <p>You can customize:</p> <ul> <li>Background colors</li> <li>Background images</li> <li>Gradients</li> <li>Videos (yes, really!)</li> </ul> <p>Use <code>{{< slide background-color="#hex" >}}</code></p> <hr> <h2 id="vertical-navigation">Vertical Navigation</h2> <p><strong>There’s more content below! ⬇️</strong></p> <p>Press the <strong>Down Arrow</strong> to see substeps.</p> <p>Note: This demonstrates Reveal.js’s vertical slide feature. Great for optional details or deep dives.</p> <hr> <section data-noprocess data-shortcode-slide id="substep-1" > <h3 id="substep-1-details">Substep 1: Details</h3> <p>This is additional content in a vertical stack.</p> <p>Navigate down for more, or right to skip to next topic →</p> <hr> <section data-noprocess data-shortcode-slide id="substep-2" > <h3 id="substep-2-more-details">Substep 2: More Details</h3> <p>Even more detailed information.</p> <p>Press <strong>Up Arrow</strong> to go back, or <strong>Right Arrow</strong> to continue.</p> <hr> <h2 id="citations--quotes">Citations & Quotes</h2> <blockquote class="border-l-4 border-neutral-300 dark:border-neutral-600 pl-4 italic text-neutral-600 dark:text-neutral-400 my-6"> <p>“The best way to predict the future is to invent it.”</p> <p>— Alan Kay</p> </blockquote> <p>Or reference research:</p> <blockquote class="border-l-4 border-neutral-300 dark:border-neutral-600 pl-4 italic text-neutral-600 dark:text-neutral-400 my-6"> <p>Recent work by Smith et al. (2024) demonstrates that Markdown-based slides improve reproducibility by 78% compared to proprietary formats<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>.</p> </blockquote> <hr> <h2 id="media-youtube-videos">Media: YouTube Videos</h2> <div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;"> </div> <p>Note: Embed YouTube videos with just the video ID. Perfect for demos, tutorials, or interviews.</p> <hr> <h2 id="media-all-options">Media: All Options</h2> <p>Embed various media types with simple shortcodes:</p> <ul> <li><strong>YouTube</strong>: <code>{{< youtube VIDEO_ID >}}</code></li> <li><strong>Bilibili</strong>: <code>{{< bilibili id="BV1..." >}}</code></li> <li><strong>Local videos</strong>: <code>{{< video src="file.mp4" controls="yes" >}}</code></li> <li><strong>Audio</strong>: <code>{{< audio src="file.mp3" >}}</code></li> </ul> <p>Perfect for demos, interviews, tutorials, or podcasts!</p> <p>Note: All media types work seamlessly in slides. Just use the appropriate shortcode.</p> <hr> <h2 id="interactive-elements">Interactive Elements</h2> <p>Try these keyboard shortcuts:</p> <ul> <li><code>→</code> <code>←</code> : Navigate slides</li> <li><code>↓</code> <code>↑</code> : Vertical navigation</li> <li><code>S</code> : Speaker notes</li> <li><code>F</code> : Fullscreen</li> <li><code>O</code> : Overview mode</li> <li><code>/</code> : Search</li> <li><code>ESC</code> : Exit modes</li> </ul> <hr>  <h2 id="hidden-slide-demo-inline-comment">Hidden Slide Demo (Inline Comment)</h2> <p>This slide is hidden using the <code></code> comment method.</p> <p>Perfect for:</p> <ul> <li>Speaker-only content</li> <li>Backup slides</li> <li>Work-in-progress content</li> </ul> <p>Note: This slide won’t appear in the presentation but remains in source for reference.</p> <hr> <h2 id="thanks">Thanks</h2> <h3 id="questions">Questions?</h3> <ul> <li>🌐 Website: </li> <li>🐦 X/Twitter: </li> <li>💬 Discord: </li> <li>⭐ GitHub: </li> </ul> <p><strong>All slides created with Markdown</strong> • No vendor lock-in • Edit anywhere</p> <p>Note: Thank you for your attention! Feel free to reach out with questions or contributions.</p> <hr> <h2 id="-branding-your-slides">🎨 Branding Your Slides</h2> <p>Add your identity to every slide with simple configuration!</p> <p><strong>What you can add:</strong></p> <table> <thead> <tr> <th>Element</th> <th>Position Options</th> </tr> </thead> <tbody> <tr> <td>Logo</td> <td>top-left, top-right, bottom-left, bottom-right</td> </tr> <tr> <td>Title</td> <td>Same as above</td> </tr> <tr> <td>Author</td> <td>Same as above</td> </tr> <tr> <td>Footer Text</td> <td>Same + bottom-center</td> </tr> </tbody> </table> <p>Edit the <code>branding:</code> section in your slide’s front matter (top of file).</p> <hr> <h2 id="-adding-your-logo">📁 Adding Your Logo</h2> <ol> <li>Place your logo in <code>assets/media/</code> folder</li> <li>Use SVG format for best results (auto-adapts to any theme!)</li> <li>Add to front matter:</li> </ol> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">branding</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">logo</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">filename</span><span class="p">:</span><span class="w"> </span><span class="s2">"your-logo.svg"</span><span class="w"> </span><span class="c"># Must be in assets/media/</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">position</span><span class="p">:</span><span class="w"> </span><span class="s2">"top-right"</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">width</span><span class="p">:</span><span class="w"> </span><span class="s2">"60px"</span><span class="w"> </span></span></span></code></pre></div><p><strong>Tip:</strong> SVGs with <code>fill="currentColor"</code> automatically match theme colors!</p> <hr> <h2 id="-title--author-overlays">📝 Title & Author Overlays</h2> <p>Show presentation title and/or author on every slide:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">branding</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">title</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">show</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">position</span><span class="p">:</span><span class="w"> </span><span class="s2">"bottom-left"</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">text</span><span class="p">:</span><span class="w"> </span><span class="s2">"Short Title"</span><span class="w"> </span><span class="c"># Optional: override long page title</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">author</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">show</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">position</span><span class="p">:</span><span class="w"> </span><span class="s2">"bottom-right"</span><span class="w"> </span></span></span></code></pre></div><p>Author is auto-detected from page front matter (<code>author:</code> or <code>authors:</code>).</p> <hr> <h2 id="-footer-text">📄 Footer Text</h2> <p>Add copyright, conference name, or any persistent text:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">branding</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">footer</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">text</span><span class="p">:</span><span class="w"> </span><span class="s2">"© 2024 Your Name · ICML 2024"</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">position</span><span class="p">:</span><span class="w"> </span><span class="s2">"bottom-center"</span><span class="w"> </span></span></span></code></pre></div><p><strong>Tip:</strong> Supports Markdown! Use <code>[Link](url)</code> for clickable links.</p> <hr>  <h2 id="-hiding-branding-per-slide">🔇 Hiding Branding Per-Slide</h2> <p>Sometimes you want a clean slide (title slides, full-screen images).</p> <p>Add this comment at the <strong>start</strong> of your slide content:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="gu">## My Clean Slide </span></span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">Content here... </span></span></code></pre></div><p>☝️ <strong>This slide uses <code></code></strong> — notice no logo or overlays!</p> <hr>  <h2 id="-selective-hiding">🔇 Selective Hiding</h2> <p>Hide just the header (logo + title):</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"> </span></span></code></pre></div><p>Or just the footer (author + footer text):</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"> </span></span></code></pre></div><p>☝️ <strong>This slide uses <code></code></strong> — footer still visible below!</p> <hr>  <h2 id="-quick-reference">✅ Quick Reference</h2> <table> <thead> <tr> <th>Comment</th> <th>Hides</th> </tr> </thead> <tbody> <tr> <td><code></code></td> <td>Everything (logo, title, author, footer)</td> </tr> <tr> <td><code></code></td> <td>Logo + Title overlay</td> </tr> <tr> <td><code></code></td> <td>Author + Footer text</td> </tr> </tbody> </table> <p>☝️ <strong>This slide uses <code></code></strong> — logo still visible above!</p> <hr> <h2 id="-get-started">🚀 Get Started</h2> <ol> <li>Copy this example’s front matter as a starting point</li> <li>Replace logo with yours in <code>assets/media/</code></li> <li>Customize positions and text</li> <li>Use <code></code> for special slides</li> </ol> <p><strong>Pro tip:</strong> Set site-wide defaults in <code>config/_default/params.yaml</code> under <code>slides.branding</code>!</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Smith, J. et al. (2024). <em>Open Science Presentations</em>. Nature Methods. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">↩︎</a></p> </li> </ol> </div> </article> <article> <h1>TransReg: Cross-transformer as the Auto-registration Module for Multi-view Mammogram Mass Detection</h1> <p>Fri, 01 Dec 2023 00:00:00 +0000</p> </article> <article> <h1>Embed Media</h1> <p>Tue, 24 Oct 2023 00:00:00 +0000</p> <p> is designed to give technical content creators a seamless experience. You can focus on the content and the HugoBlox Kit which this template is built upon handles the rest.</p> <p><strong>Embed videos, podcasts, code, LaTeX math, and even test students!</strong></p> <p>On this page, you’ll find some examples of the types of technical content that can be rendered with Hugo Blox.</p> <h2 id="video">Video</h2> <p>Teach your course by sharing videos with your students. Choose from one of the following approaches:</p> <div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;"> </div> <p><strong>Youtube</strong>:</p> <pre><code>{{< youtube w7Ft2ymGmfc >}} </code></pre> <p><strong>Bilibili</strong>:</p> <pre><code>{{< bilibili id="BV1WV4y1r7DF" >}} </code></pre> <p><strong>Video file</strong></p> <p>Videos may be added to a page by either placing them in your <code>assets/media/</code> media library or in your , and then embedding them with the <em>video</em> shortcode:</p> <pre><code>{{< video src="my_video.mp4" controls="yes" >}} </code></pre> <h2 id="podcast">Podcast</h2> <p>You can add a podcast or music to a page by placing the MP3 file in the page’s folder or the media library folder and then embedding the audio on your page with the <em>audio</em> shortcode:</p> <pre><code>{{< audio src="ambient-piano.mp3" >}} </code></pre> <p>Try it out:</p> </audio> <h2 id="test-students">Test students</h2> <p>Provide a simple yet fun self-assessment by revealing the solutions to challenges with the <code>spoiler</code> shortcode:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl">{{< spoiler text="👉 Click to view the solution" >}} </span></span><span class="line"><span class="cl">You found me! </span></span><span class="line"><span class="cl">{{< /spoiler >}} </span></span></code></pre></div><p>renders as</p> <details class="spoiler " id="spoiler-2"> <summary class="cursor-pointer">👉 Click to view the solution</summary> <div class="rounded-lg bg-neutral-50 dark:bg-neutral-800 p-2"> You found me 🎉 </div> </details> <h2 id="math">Math</h2> <p>HugoBlox Kit supports a Markdown extension for $\LaTeX$ math. You can enable this feature by toggling the <code>math</code> option in your <code>config/_default/params.yaml</code> file.</p> <p>To render <em>inline</em> or <em>block</em> math, wrap your LaTeX math with <code>{{< math >}}$...${{< /math >}}</code> or <code>{{< math >}}$$...$${{< /math >}}</code>, respectively.</p> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-blue-100 dark:bg-blue-900 border-blue-500" data-callout="note" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-blue-600 dark:text-blue-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="m16.862 4.487l1.687-1.688a1.875 1.875 0 1 1 2.652 2.652L6.832 19.82a4.5 4.5 0 0 1-1.897 1.13l-2.685.8l.8-2.685a4.5 4.5 0 0 1 1.13-1.897zm0 0L19.5 7.125"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Note</div> <div class="callout-body"><p>We wrap the LaTeX math in the Hugo Blox <em>math</em> shortcode to prevent Hugo rendering our math as Markdown. This callout now uses the standard Markdown alert syntax!</p></div> </div> </div> <p>Example <strong>math block</strong>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-latex" data-lang="latex"><span class="line"><span class="cl"><span class="nb">{{</span>< math ><span class="nb">}}</span> </span></span><span class="line"><span class="cl"><span class="sb">$$</span><span class="nb"> </span></span></span><span class="line"><span class="cl"><span class="nv">\gamma</span><span class="nb">_{n} </span><span class="o">=</span><span class="nb"> </span><span class="nv">\frac</span><span class="nb">{ </span><span class="nv">\left</span><span class="nb"> | </span><span class="nv">\left</span><span class="nb"> </span><span class="o">(</span><span class="nv">\mathbf</span><span class="nb"> x_{n} </span><span class="o">-</span><span class="nb"> </span><span class="nv">\mathbf</span><span class="nb"> x_{n</span><span class="o">-</span><span class="m">1</span><span class="nb">} </span><span class="nv">\right</span><span class="nb"> </span><span class="o">)</span><span class="nb">^T </span><span class="nv">\left</span><span class="nb"> </span><span class="o">[</span><span class="nv">\nabla</span><span class="nb"> F </span><span class="o">(</span><span class="nv">\mathbf</span><span class="nb"> x_{n}</span><span class="o">)</span><span class="nb"> </span><span class="o">-</span><span class="nb"> </span><span class="nv">\nabla</span><span class="nb"> F </span><span class="o">(</span><span class="nv">\mathbf</span><span class="nb"> x_{n</span><span class="o">-</span><span class="m">1</span><span class="nb">}</span><span class="o">)</span><span class="nb"> </span><span class="nv">\right</span><span class="nb"> </span><span class="o">]</span><span class="nb"> </span><span class="nv">\right</span><span class="nb"> |}{</span><span class="nv">\left</span><span class="nb"> </span><span class="nv">\|\nabla</span><span class="nb"> F</span><span class="o">(</span><span class="nv">\mathbf</span><span class="nb">{x}_{n}</span><span class="o">)</span><span class="nb"> </span><span class="o">-</span><span class="nb"> </span><span class="nv">\nabla</span><span class="nb"> F</span><span class="o">(</span><span class="nv">\mathbf</span><span class="nb">{x}_{n</span><span class="o">-</span><span class="m">1</span><span class="nb">}</span><span class="o">)</span><span class="nb"> </span><span class="nv">\right</span><span class="nb"> </span><span class="nv">\|</span><span class="nb">^</span><span class="m">2</span><span class="nb">} </span></span></span><span class="line"><span class="cl"><span class="s">$$</span> </span></span><span class="line"><span class="cl"><span class="nb">{{</span>< /math ><span class="nb">}}</span> </span></span></code></pre></div><p>renders as</p> $$\gamma_{n} = \frac{ \left | \left (\mathbf x_{n} - \mathbf x_{n-1} \right )^T \left [\nabla F (\mathbf x_{n}) - \nabla F (\mathbf x_{n-1}) \right ] \right |}{\left \|\nabla F(\mathbf{x}_{n}) - \nabla F(\mathbf{x}_{n-1}) \right \|^2}$$ <p>Example <strong>inline math</strong> <code>{{< math >}}$\nabla F(\mathbf{x}_{n})${{< /math >}}</code> renders as $\nabla F(\mathbf{x}_{n})$ .</p> <p>Example <strong>multi-line math</strong> using the math linebreak (<code>\\</code>):</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-latex" data-lang="latex"><span class="line"><span class="cl"><span class="nb">{{</span>< math ><span class="nb">}}</span> </span></span><span class="line"><span class="cl"><span class="sb">$$</span><span class="nb">f</span><span class="o">(</span><span class="nb">k;p_{</span><span class="m">0</span><span class="nb">}^{</span><span class="o">*</span><span class="nb">}</span><span class="o">)</span><span class="nb"> </span><span class="o">=</span><span class="nb"> </span><span class="nv">\begin</span><span class="nb">{cases}p_{</span><span class="m">0</span><span class="nb">}^{</span><span class="o">*</span><span class="nb">} & </span><span class="nv">\text</span><span class="nb">{if }k</span><span class="o">=</span><span class="m">1</span><span class="nb">, </span><span class="nv">\\</span><span class="nb"> </span></span></span><span class="line"><span class="cl"><span class="m">1</span><span class="o">-</span><span class="nb">p_{</span><span class="m">0</span><span class="nb">}^{</span><span class="o">*</span><span class="nb">} & </span><span class="nv">\text</span><span class="nb">{if }k</span><span class="o">=</span><span class="m">0</span><span class="nb">.</span><span class="nv">\end</span><span class="nb">{cases}</span><span class="s">$$</span> </span></span><span class="line"><span class="cl"><span class="nb">{{</span>< /math ><span class="nb">}}</span> </span></span></code></pre></div><p>renders as</p> $$ f(k;p_{0}^{*}) = \begin{cases}p_{0}^{*} & \text{if }k=1, \\ 1-p_{0}^{*} & \text{if }k=0.\end{cases} $$ <h2 id="code">Code</h2> <p>HugoBlox Kit utilises Hugo’s Markdown extension for highlighting code syntax. The code theme can be selected in the <code>config/_default/params.yaml</code> file.</p> <pre><code>```python import pandas as pd data = pd.read_csv("data.csv") data.head() ``` </code></pre> <p>renders as</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span> </span></span><span class="line"><span class="cl"><span class="n">data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s2">"data.csv"</span><span class="p">)</span> </span></span><span class="line"><span class="cl"><span class="n">data</span><span class="o">.</span><span class="n">head</span><span class="p">()</span> </span></span></code></pre></div><h2 id="inline-images">Inline Images</h2> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="p">{{<</span><span class="w"> </span><span class="nx">icon</span><span class="w"> </span><span class="nx">name</span><span class="p">=</span><span class="s">"python"</span><span class="w"> </span><span class="p">>}}</span><span class="w"> </span><span class="nx">Python</span><span class="w"> </span></span></span></code></pre></div><p>renders as</p> <p> <span class="inline-block pr-1"> <svg style="height: 1em; transform: translateY(0.1em);" xmlns="http://www.w3.org/2000/svg" height="1em" viewBox="0 0 448 512" fill="currentColor"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> </span> Python</p> <h2 id="did-you-find-this-page-helpful-consider-sharing-it-">Did you find this page helpful? Consider sharing it 🙌</h2> </article> <article> <h1>Experience</h1> <p>Tue, 24 Oct 2023 00:00:00 +0000</p> </article> <article> <h1>A Novel Transparency Strategy-based Data Augmentation Approach for BI-RADS Classification of Mammograms</h1> <p>Sun, 01 Jan 2023 00:00:00 +0000</p> </article> <article> <h1>Exploring Cultural Alignment and Bias in Large Language Model: A Vietnamese Contextual Study</h1> <p>Sun, 01 Jan 2023 00:00:00 +0000</p> </article> <article> <h1>Buttons</h1> <p>Mon, 01 Jan 0001 00:00:00 +0000</p> <p>A modern, customizable button shortcode with gradient styling, icons, and smart link handling.</p> <h2 id="basic-usage">Basic Usage</h2> <div class="text-left"> <a id="button-351ac412531c23e422b81c6422c17da5" href="https://chiphan1110.github.io/contact" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Contact Us" > <span>Contact Us</span> </a> </div> <div class="text-left"> <a id="button-99f90e9d2b754ffe3fc7b4f9d280ab9e" href="https://example.com" target="_blank" rel="noopener noreferrer" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-secondary-500 to-secondary-600 hover:from-secondary-600 hover:to-secondary-700 active:from-secondary-700 active:to-secondary-800 text-white shadow-lg shadow-secondary-500/25 hover:shadow-xl hover:shadow-secondary-500/30 hover:scale-105 active:scale-95 focus:ring-secondary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Visit External Site" > <span>Visit External Site</span> </a> </div> <p>The above buttons are created with:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"/contact"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Contact Us<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"https://example.com"</span><span class="w"> </span><span class="nx">new_tab</span><span class="o">=</span><span class="s">"true"</span><span class="w"> </span><span class="nx">style</span><span class="o">=</span><span class="s">"secondary"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Visit External Site<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h2 id="style-variants">Style Variants</h2> <h3 id="primary-default">Primary (Default)</h3> <div class="text-left"> <a id="button-33f0c5949dceb4f2158940543a6d11f0" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Primary Button" > <span>Primary Button</span> </a> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">style</span><span class="o">=</span><span class="s">"primary"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Primary Button<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h3 id="secondary">Secondary</h3> <div class="text-left"> <a id="button-654c665d9c02125e00b3c2638e02ff31" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-secondary-500 to-secondary-600 hover:from-secondary-600 hover:to-secondary-700 active:from-secondary-700 active:to-secondary-800 text-white shadow-lg shadow-secondary-500/25 hover:shadow-xl hover:shadow-secondary-500/30 hover:scale-105 active:scale-95 focus:ring-secondary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Secondary Button" > <span>Secondary Button</span> </a> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">style</span><span class="o">=</span><span class="s">"secondary"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Secondary Button<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h3 id="outline">Outline</h3> <div class="text-left"> <a id="button-569954ebe8d760dfbd5f8dd8d491e972" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-white dark:bg-zinc-900 border-2 border-primary-500 text-primary-600 dark:text-primary-400 hover:bg-primary-50 dark:hover:bg-primary-950/50 hover:border-primary-600 active:bg-primary-100 dark:active:bg-primary-950 shadow-md hover:shadow-lg hover:scale-105 active:scale-95 focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Outline Button" > <span>Outline Button</span> </a> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">style</span><span class="o">=</span><span class="s">"outline"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Outline Button<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h3 id="ghost">Ghost</h3> <div class="text-left"> <a id="button-e8d694dcc82ccc6b692376347eb1210c" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-transparent text-primary-600 dark:text-primary-400 hover:bg-primary-50 dark:hover:bg-primary-950/50 active:bg-primary-100 dark:active:bg-primary-950 hover:scale-105 active:scale-95 focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Ghost Button" > <span>Ghost Button</span> </a> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">style</span><span class="o">=</span><span class="s">"ghost"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Ghost Button<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h2 id="sizes">Sizes</h2> <h3 id="small">Small</h3> <div class="text-left"> <a id="button-9edd9a4164984b892472dc4cb0d94acb" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-3 py-1.5 text-sm rounded-lg" role="button" aria-label="Small Button" > <span>Small Button</span> </a> </div> <h3 id="medium-default">Medium (Default)</h3> <div class="text-left"> <a id="button-917c91b5df194ee244bcb8dd7d4916c5" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Medium Button" > <span>Medium Button</span> </a> </div> <h3 id="large">Large</h3> <div class="text-left"> <a id="button-c9ef1eeebbeac0de80c5666baeda508c" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-6 py-3 text-lg rounded-lg" role="button" aria-label="Large Button" > <span>Large Button</span> </a> </div> <h3 id="extra-large">Extra Large</h3> <div class="text-left"> <a id="button-c38da6a61bc9a2511c12a071b05d1527" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-8 py-4 text-xl rounded-lg" role="button" aria-label="Extra Large" > <span>Extra Large</span> </a> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">size</span><span class="o">=</span><span class="s">"sm"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Small Button<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">size</span><span class="o">=</span><span class="s">"md"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Medium Button<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">size</span><span class="o">=</span><span class="s">"lg"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Large Button<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">size</span><span class="o">=</span><span class="s">"xl"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Extra Large<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h2 id="alignment">Alignment</h2> <h3 id="left-default">Left (Default)</h3> <div class="text-left"> <a id="button-46275a2130718485193b07442854e305" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Left Aligned" > <span>Left Aligned</span> </a> </div> <h3 id="center">Center</h3> <div class="text-center"> <a id="button-2e7c6d0545131be865f91f4dcb47ebd3" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Center Aligned" > <span>Center Aligned</span> </a> </div> <h3 id="right">Right</h3> <div class="text-right"> <a id="button-09d01d6786049ba2a63f2cafd484e60e" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Right Aligned" > <span>Right Aligned</span> </a> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">align</span><span class="o">=</span><span class="s">"left"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Left Aligned<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">align</span><span class="o">=</span><span class="s">"center"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Center Aligned<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">align</span><span class="o">=</span><span class="s">"right"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Right Aligned<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h2 id="with-icons">With Icons</h2> <h3 id="icon-before-text">Icon Before Text</h3> <div class="text-left"> <a id="button-2070b64a1b12b8165d04699a5e4f0a4d" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Download" > <span class="flex-shrink-0"> <svg class="w-4 h-4" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M3 16.5v2.25A2.25 2.25 0 0 0 5.25 21h13.5A2.25 2.25 0 0 0 21 18.75V16.5M16.5 12L12 16.5m0 0L7.5 12m4.5 4.5V3"/></svg> </span> <span>Download</span> </a> </div> <h3 id="icon-after-text">Icon After Text</h3> <div class="text-left"> <a id="button-213c46a162adc66071b38a55c26716e3" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Continue" > <span>Continue</span> <span class="flex-shrink-0"> <svg class="w-4 h-4" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M13.5 4.5L21 12m0 0l-7.5 7.5M21 12H3"/></svg> </span> </a> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">icon</span><span class="o">=</span><span class="s">"arrow-down-tray"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Download<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">icon</span><span class="o">=</span><span class="s">"arrow-right"</span><span class="w"> </span><span class="nx">icon_position</span><span class="o">=</span><span class="s">"right"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Continue<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h2 id="rounded-corners">Rounded Corners</h2> <h3 id="small-radius">Small Radius</h3> <div class="text-left"> <a id="button-f306324e9ea3297cc7149761131e0e41" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded" role="button" aria-label="Small Radius" > <span>Small Radius</span> </a> </div> <h3 id="medium-radius-default">Medium Radius (Default)</h3> <div class="text-left"> <a id="button-d519a3255417739cb8197b9869248dec" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="Medium Radius" > <span>Medium Radius</span> </a> </div> <h3 id="large-radius">Large Radius</h3> <div class="text-left"> <a id="button-920afdfab9e2ab0e88cb5046924cd223" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded-xl" role="button" aria-label="Large Radius" > <span>Large Radius</span> </a> </div> <h3 id="pill-shape">Pill Shape</h3> <div class="text-left"> <a id="button-464be1a963c8f63cf75275ea2137551a" href="#" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-4 py-2 text-base rounded-full" role="button" aria-label="Pill Button" > <span>Pill Button</span> </a> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">rounded</span><span class="o">=</span><span class="s">"sm"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Small Radius<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">rounded</span><span class="o">=</span><span class="s">"md"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Medium Radius<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">rounded</span><span class="o">=</span><span class="s">"lg"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Large Radius<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"#"</span><span class="w"> </span><span class="nx">rounded</span><span class="o">=</span><span class="s">"full"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Pill Button<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h2 id="advanced-examples">Advanced Examples</h2> <h3 id="call-to-action-button">Call-to-Action Button</h3> <div class="text-center"> <a id="button-62059f8d62052633590d3f60ec092595" href="https://chiphan1110.github.io/signup" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-primary-500 to-primary-600 hover:from-primary-600 hover:to-primary-700 active:from-primary-700 active:to-primary-800 text-white shadow-lg shadow-primary-500/25 hover:shadow-xl hover:shadow-primary-500/30 hover:-translate-y-0.5 hover:scale-[1.02] active:scale-[0.98] focus:ring-primary-500/50 px-6 py-3 text-lg rounded-lg" role="button" aria-label="Get Started Today" > <span class="flex-shrink-0"> <svg class="w-4 h-4" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M15.59 14.37q.159.666.16 1.38a6 6 0 0 1-6 6v-4.8m5.84-2.58a14.98 14.98 0 0 0 6.16-12.12A14.98 14.98 0 0 0 9.631 8.41m5.96 5.96a14.9 14.9 0 0 1-5.841 2.58m-.119-8.54a6 6 0 0 0-7.381 5.84h4.8m2.581-5.84a14.9 14.9 0 0 0-2.58 5.84m2.699 2.7q-.155.032-.311.06a15 15 0 0 1-2.448-2.448l.06-.312m-2.24 2.39a4.49 4.49 0 0 0-1.757 4.306q.341.054.696.054a4.5 4.5 0 0 0 3.61-1.812M16.5 9a1.5 1.5 0 1 1-3 0a1.5 1.5 0 0 1 3 0"/></svg> </span> <span>Get Started Today</span> </a> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"/signup"</span><span class="w"> </span><span class="nx">style</span><span class="o">=</span><span class="s">"primary"</span><span class="w"> </span><span class="nx">size</span><span class="o">=</span><span class="s">"lg"</span><span class="w"> </span><span class="nx">align</span><span class="o">=</span><span class="s">"center"</span><span class="w"> </span><span class="nx">icon</span><span class="o">=</span><span class="s">"rocket-launch"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Get Started Today<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h3 id="external-link-with-new-tab">External Link with New Tab</h3> <div class="text-left"> <a id="button-2cb1bb8bbb2590cc46a8ead690d2da77" href="https://github.com/hugo-blox/kit" target="_blank" rel="noopener noreferrer" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-white dark:bg-zinc-900 border-2 border-primary-500 text-primary-600 dark:text-primary-400 hover:bg-primary-50 dark:hover:bg-primary-950/50 hover:border-primary-600 active:bg-primary-100 dark:active:bg-primary-950 shadow-md hover:shadow-lg hover:scale-105 active:scale-95 focus:ring-primary-500/50 px-4 py-2 text-base rounded-lg" role="button" aria-label="View on GitHub" > <span>View on GitHub</span> <span class="flex-shrink-0"> <svg class="w-4 h-4" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M13.5 6H5.25A2.25 2.25 0 0 0 3 8.25v10.5A2.25 2.25 0 0 0 5.25 21h10.5A2.25 2.25 0 0 0 18 18.75V10.5m-10.5 6L21 3m0 0h-5.25M21 3v5.25"/></svg> </span> </a> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"https://github.com/hugo-blox/kit"</span><span class="w"> </span><span class="nx">new_tab</span><span class="o">=</span><span class="s">"true"</span><span class="w"> </span><span class="nx">style</span><span class="o">=</span><span class="s">"outline"</span><span class="w"> </span><span class="nx">icon</span><span class="o">=</span><span class="s">"arrow-top-right-on-square"</span><span class="w"> </span><span class="nx">icon_position</span><span class="o">=</span><span class="s">"right"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>View on GitHub<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h3 id="download-button">Download Button</h3> <div class="text-left"> <a id="button-72ec86801f391f16b84b269ade9d2995" href="https://chiphan1110.github.io/files/document.pdf" class="inline-flex items-center gap-2 font-medium no-underline transition-all duration-300 ease-out transform-gpu focus:outline-none focus:ring-4 focus:ring-offset-2 focus:ring-offset-white dark:focus:ring-offset-zinc-900 disabled:opacity-50 disabled:cursor-not-allowed disabled:pointer-events-none bg-gradient-to-br from-secondary-500 to-secondary-600 hover:from-secondary-600 hover:to-secondary-700 active:from-secondary-700 active:to-secondary-800 text-white shadow-lg shadow-secondary-500/25 hover:shadow-xl hover:shadow-secondary-500/30 hover:scale-105 active:scale-95 focus:ring-secondary-500/50 px-4 py-2 text-base rounded-full" role="button" aria-label="Download PDF" > <span class="flex-shrink-0"> <svg class="w-4 h-4" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M19.5 14.25v-2.625a3.375 3.375 0 0 0-3.375-3.375h-1.5A1.125 1.125 0 0 1 13.5 7.125v-1.5a3.375 3.375 0 0 0-3.375-3.375H8.25m.75 12l3 3m0 0l3-3m-3 3v-6m-1.5-9H5.625c-.621 0-1.125.504-1.125 1.125v17.25c0 .621.504 1.125 1.125 1.125h12.75c.621 0 1.125-.504 1.125-1.125V11.25a9 9 0 0 0-9-9"/></svg> </span> <span>Download PDF</span> </a> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go-html-template" data-lang="go-html-template"><span class="line"><span class="cl"><span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="nx">button</span><span class="w"> </span><span class="nx">url</span><span class="o">=</span><span class="s">"/files/document.pdf"</span><span class="w"> </span><span class="nx">style</span><span class="o">=</span><span class="s">"secondary"</span><span class="w"> </span><span class="nx">icon</span><span class="o">=</span><span class="s">"document-arrow-down"</span><span class="w"> </span><span class="nx">rounded</span><span class="o">=</span><span class="s">"full"</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span>Download PDF<span class="cp">{{</span><span class="err"><</span><span class="w"> </span><span class="err">/</span><span class="nx">button</span><span class="w"> </span><span class="err">></span><span class="cp">}}</span> </span></span></code></pre></div><h2 id="parameters">Parameters</h2> <table> <thead> <tr> <th>Parameter</th> <th>Type</th> <th>Default</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td><code>url</code></td> <td>string</td> <td><code>#</code></td> <td><strong>Required.</strong> Button destination URL (internal or external)</td> </tr> <tr> <td><code>text</code></td> <td>string</td> <td>Inner content</td> <td>Button text (overrides shortcode content)</td> </tr> <tr> <td><code>new_tab</code></td> <td>boolean</td> <td><code>false</code></td> <td>Whether to open link in new tab</td> </tr> <tr> <td><code>style</code></td> <td>string</td> <td><code>primary</code></td> <td>Button style: <code>primary</code>, <code>secondary</code>, <code>outline</code>, <code>ghost</code></td> </tr> <tr> <td><code>size</code></td> <td>string</td> <td><code>md</code></td> <td>Button size: <code>sm</code>, <code>md</code>, <code>lg</code>, <code>xl</code></td> </tr> <tr> <td><code>align</code></td> <td>string</td> <td><code>left</code></td> <td>Button alignment: <code>left</code>, <code>center</code>, <code>right</code></td> </tr> <tr> <td><code>icon</code></td> <td>string</td> <td>-</td> <td>Icon name from </td> </tr> <tr> <td><code>icon_position</code></td> <td>string</td> <td><code>left</code></td> <td>Icon position: <code>left</code>, <code>right</code></td> </tr> <tr> <td><code>rounded</code></td> <td>string</td> <td><code>md</code></td> <td>Border radius: <code>sm</code>, <code>md</code>, <code>lg</code>, <code>xl</code>, <code>full</code></td> </tr> <tr> <td><code>disabled</code></td> <td>boolean</td> <td><code>false</code></td> <td>Whether button should be disabled</td> </tr> </tbody> </table> <h2 id="security-features">Security Features</h2> <p>The button shortcode automatically handles security for external links:</p> <ul> <li><strong>External links</strong> get <code>rel="noreferrer"</code> attribute</li> <li><strong>External links opening in new tab</strong> get <code>rel="noopener noreferrer"</code></li> <li><strong>Internal links opening in new tab</strong> get <code>rel="noopener"</code></li> </ul> <p>This ensures safe navigation while maintaining functionality.</p> <h2 id="accessibility">Accessibility</h2> <p>The button shortcode includes built-in accessibility features:</p> <ul> <li>Proper <code>role="button"</code> attribute</li> <li><code>aria-label</code> support</li> <li>Keyboard focus indicators</li> <li>High contrast focus rings</li> <li>Disabled state handling</li> </ul> </article> <article> <h1>Callouts</h1> <p>Mon, 01 Jan 0001 00:00:00 +0000</p> <p>Hugo Blox supports GitHub and Obsidian-style Markdown callouts for maximum compatibility and content portability.</p> <p>Callouts are a useful feature to draw attention to important or related content such as notes, hints, or warnings in your articles.</p> <h2 id="usage">Usage</h2> <h3 id="basic-callout-types">Basic Callout Types</h3> <p>Hugo Blox supports all 15+ callout types from :</p> <p><strong>Information & Notes:</strong></p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!NOTE] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">This is a note callout with important information that users should know. </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-blue-100 dark:bg-blue-900 border-blue-500" data-callout="note" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-blue-600 dark:text-blue-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="m16.862 4.487l1.687-1.688a1.875 1.875 0 1 1 2.652 2.652L6.832 19.82a4.5 4.5 0 0 1-1.897 1.13l-2.685.8l.8-2.685a4.5 4.5 0 0 1 1.13-1.897zm0 0L19.5 7.125"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Note</div> <div class="callout-body"><p>This is a note callout with important information that users should know.</p></div> </div> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!INFO] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">Alternative info callout - same styling as NOTE. </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-blue-100 dark:bg-blue-900 border-blue-500" data-callout="info" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-blue-600 dark:text-blue-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="m11.25 11.25l.041-.02a.75.75 0 0 1 1.063.852l-.708 2.836a.75.75 0 0 0 1.063.853l.041-.021M21 12a9 9 0 1 1-18 0a9 9 0 0 1 18 0m-9-3.75h.008v.008H12z"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Info</div> <div class="callout-body"><p>Alternative info callout - same styling as NOTE.</p></div> </div> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!ABSTRACT] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">Use for abstracts, summaries, or TL;DR sections. </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-cyan-100 dark:bg-cyan-900 border-cyan-500" data-callout="abstract" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-cyan-600 dark:text-cyan-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M9 12h3.75M9 15h3.75M9 18h3.75m3 .75H18a2.25 2.25 0 0 0 2.25-2.25V6.108c0-1.135-.845-2.098-1.976-2.192a48 48 0 0 0-1.123-.08m-5.801 0q-.099.316-.1.664c0 .414.336.75.75.75h4.5a.75.75 0 0 0 .75-.75a2.3 2.3 0 0 0-.1-.664m-5.8 0A2.25 2.25 0 0 1 13.5 2.25H15a2.25 2.25 0 0 1 2.15 1.586m-5.8 0q-.563.035-1.124.08C9.095 4.01 8.25 4.973 8.25 6.108V8.25m0 0H4.875c-.621 0-1.125.504-1.125 1.125v11.25c0 .621.504 1.125 1.125 1.125h9.75c.621 0 1.125-.504 1.125-1.125V9.375c0-.621-.504-1.125-1.125-1.125zM6.75 12h.008v.008H6.75zm0 3h.008v.008H6.75zm0 3h.008v.008H6.75z"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Abstract</div> <div class="callout-body"><p>Use for abstracts, summaries, or TL;DR sections.</p></div> </div> </div> <p><strong>Actions & Tasks:</strong></p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!TODO] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">This is something that needs to be done. </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-blue-100 dark:bg-blue-900 border-blue-500" data-callout="todo" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-blue-600 dark:text-blue-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M9 12.75L11.25 15L15 9.75M21 12a9 9 0 1 1-18 0a9 9 0 0 1 18 0"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Todo</div> <div class="callout-body"><p>This is something that needs to be done.</p></div> </div> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!TIP] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">Here's a helpful tip to make your workflow more efficient! </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-emerald-100 dark:bg-emerald-900 border-emerald-500" data-callout="tip" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-emerald-600 dark:text-emerald-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M12 18v-5.25m0 0a6 6 0 0 0 1.5-.189m-1.5.189a6 6 0 0 1-1.5-.189m3.75 7.478a12.1 12.1 0 0 1-4.5 0m3.75 2.383a14.4 14.4 0 0 1-3 0M14.25 18v-.192c0-.983.658-1.823 1.508-2.316a7.5 7.5 0 1 0-7.517 0c.85.493 1.509 1.333 1.509 2.316V18"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Tip</div> <div class="callout-body"><p>Here’s a helpful tip to make your workflow more efficient!</p></div> </div> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!SUCCESS] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">Great job! This operation completed successfully. </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-green-100 dark:bg-green-900 border-green-500" data-callout="success" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-green-600 dark:text-green-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M9 12.75L11.25 15L15 9.75M21 12a9 9 0 1 1-18 0a9 9 0 0 1 18 0"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Success</div> <div class="callout-body"><p>Great job! This operation completed successfully.</p></div> </div> </div> <p><strong>Questions & Interactive:</strong></p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!QUESTION] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">What do you think about this approach? Let us know! </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-yellow-100 dark:bg-yellow-900 border-yellow-500" data-callout="question" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-yellow-700 dark:text-yellow-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M9.879 7.519c1.172-1.025 3.071-1.025 4.243 0c1.171 1.025 1.171 2.687 0 3.712q-.308.268-.67.442c-.746.361-1.452.999-1.452 1.827v.75M21 12a9 9 0 1 1-18 0a9 9 0 0 1 18 0m-9 5.25h.008v.008H12z"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Question</div> <div class="callout-body"><p>What do you think about this approach? Let us know!</p></div> </div> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!EXAMPLE] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">Here's a practical example of how to implement this feature. </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-purple-100 dark:bg-purple-900 border-purple-500" data-callout="example" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-purple-600 dark:text-purple-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M9.75 3.104v5.714a2.25 2.25 0 0 1-.659 1.591L5 14.5M9.75 3.104q-.376.034-.75.082m.75-.082a24.3 24.3 0 0 1 4.5 0m0 0v5.714c0 .597.237 1.17.659 1.591L19.8 15.3M14.25 3.104q.377.034.75.082M19.8 15.3l-1.57.393A9.07 9.07 0 0 1 12 15a9.07 9.07 0 0 0-6.23-.693L5 14.5m14.8.8l1.402 1.402c1.232 1.232.65 3.318-1.067 3.611A48.3 48.3 0 0 1 12 21a48 48 0 0 1-8.135-.687c-1.718-.293-2.3-2.379-1.067-3.61L5 14.5"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Example</div> <div class="callout-body"><p>Here’s a practical example of how to implement this feature.</p></div> </div> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!QUOTE] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">"The best way to predict the future is to invent it." - Alan Kay </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-gray-100 dark:bg-gray-800 border-gray-500" data-callout="quote" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-gray-600 dark:text-gray-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M20.25 8.511c.884.284 1.5 1.128 1.5 2.097v4.286c0 1.136-.847 2.1-1.98 2.193q-.51.041-1.02.072v3.091l-3-3q-2.031 0-4.02-.163a2.1 2.1 0 0 1-.825-.242m9.345-8.334a2 2 0 0 0-.476-.095a48.6 48.6 0 0 0-8.048 0c-1.131.094-1.976 1.057-1.976 2.192v4.286c0 .837.46 1.58 1.155 1.951m9.345-8.334V6.637c0-1.621-1.152-3.026-2.76-3.235A48.5 48.5 0 0 0 11.25 3c-2.115 0-4.198.137-6.24.402c-1.608.209-2.76 1.614-2.76 3.235v6.226c0 1.621 1.152 3.026 2.76 3.235q.865.113 1.74.194V21l4.155-4.155"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Quote</div> <div class="callout-body"><p>“The best way to predict the future is to invent it.” - Alan Kay</p></div> </div> </div> <p><strong>Warnings & Errors:</strong></p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!WARNING] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">Be careful! This action might have unexpected consequences. </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-orange-100 dark:bg-orange-900 border-orange-500" data-callout="warning" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-orange-600 dark:text-orange-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M12 9v3.75m-9.303 3.376c-.866 1.5.217 3.374 1.948 3.374h14.71c1.73 0 2.813-1.874 1.948-3.374L13.949 3.378c-.866-1.5-3.032-1.5-3.898 0zM12 15.75h.007v.008H12z"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Warning</div> <div class="callout-body"><p>Be careful! This action might have unexpected consequences.</p></div> </div> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!CAUTION] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">Danger! This operation is irreversible and could cause data loss. </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-red-100 dark:bg-red-900 border-red-500" data-callout="caution" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-red-600 dark:text-red-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M12 9v3.75m-9.303 3.376c-.866 1.5.217 3.374 1.948 3.374h14.71c1.73 0 2.813-1.874 1.948-3.374L13.949 3.378c-.866-1.5-3.032-1.5-3.898 0zM12 15.75h.007v.008H12z"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Caution</div> <div class="callout-body"><p>Danger! This operation is irreversible and could cause data loss.</p></div> </div> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!IMPORTANT] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">This is critical information that users must understand to proceed. </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-purple-100 dark:bg-purple-900 border-purple-500" data-callout="important" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-purple-600 dark:text-purple-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M12 9v3.75m9-.75a9 9 0 1 1-18 0a9 9 0 0 1 18 0m-9 3.75h.008v.008H12z"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Important</div> <div class="callout-body"><p>This is critical information that users must understand to proceed.</p></div> </div> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!DANGER] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">This is extremely dangerous - proceed with extreme caution! </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-red-100 dark:bg-red-900 border-red-500" data-callout="danger" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-red-600 dark:text-red-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M12 9v3.75m-9.303 3.376c-.866 1.5.217 3.374 1.948 3.374h14.71c1.73 0 2.813-1.874 1.948-3.374L13.949 3.378c-.866-1.5-3.032-1.5-3.898 0zM12 15.75h.007v.008H12z"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Danger</div> <div class="callout-body"><p>This is extremely dangerous - proceed with extreme caution!</p></div> </div> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!FAILURE] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">This operation failed. Check your configuration and try again. </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-red-100 dark:bg-red-900 border-red-500" data-callout="failure" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-red-600 dark:text-red-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="m9.75 9.75l4.5 4.5m0-4.5l-4.5 4.5M21 12a9 9 0 1 1-18 0a9 9 0 0 1 18 0"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Failure</div> <div class="callout-body"><p>This operation failed. Check your configuration and try again.</p></div> </div> </div> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!BUG] </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">Known issue: This feature doesn't work properly in Safari < 14. </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-red-100 dark:bg-red-900 border-red-500" data-callout="bug" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-red-600 dark:text-red-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M12 12.75q1.724 0 3.383.237c1.037.146 1.866.966 1.866 2.013c0 3.728-2.35 6.75-5.25 6.75S6.75 18.728 6.75 15c0-1.046.83-1.867 1.866-2.013A24 24 0 0 1 12 12.75m0 0c2.883 0 5.647.508 8.208 1.44a24 24 0 0 1-1.153 6.06M12 12.75c-2.883 0-5.647.508-8.208 1.44c.125 2.105.52 4.136 1.153 6.06M12 12.75a2.25 2.25 0 0 0 2.248-2.354M12 12.75a2.25 2.25 0 0 1-2.248-2.354M12 8.25q1.494-.001 2.922-.236c.403-.066.74-.358.795-.762a3.8 3.8 0 0 0-.399-2.25M12 8.25q-1.493-.001-2.922-.236c-.402-.066-.74-.358-.795-.762a3.73 3.73 0 0 1 .4-2.253M12 8.25a2.25 2.25 0 0 0-2.248 2.146M12 8.25a2.25 2.25 0 0 1 2.248 2.146M8.683 5a6 6 0 0 1-1.155-1.002c.07-.63.27-1.222.574-1.747M8.683 5a3.75 3.75 0 0 1 6.635 0m0 0c.427-.283.815-.62 1.155-.999a4.5 4.5 0 0 0-.575-1.752M4.921 6a24 24 0 0 0-.392 3.314a24 24 0 0 0 5.223 1.082M19.08 6q.308 1.622.392 3.314a24 24 0 0 1-5.223 1.082"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Bug</div> <div class="callout-body"><p>Known issue: This feature doesn’t work properly in Safari < 14.</p></div> </div> </div> <h3 id="custom-titles">Custom Titles</h3> <p>You can customize the title of any callout:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl"><span class="k">> </span><span class="ge">[!WARNING]+ Custom Warning Title </span></span></span><span class="line"><span class="cl"><span class="k">> </span><span class="ge">This warning has a custom title instead of just "Warning". </span></span></span></code></pre></div> <div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-orange-100 dark:bg-orange-900 border-orange-500" data-callout="warning" data-callout-metadata=""> <span class="callout-icon pr-3 pt-1 text-orange-600 dark:text-orange-300"> <svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M12 9v3.75m-9.303 3.376c-.866 1.5.217 3.374 1.948 3.374h14.71c1.73 0 2.813-1.874 1.948-3.374L13.949 3.378c-.866-1.5-3.032-1.5-3.898 0zM12 15.75h.007v.008H12z"/></svg> </span> <div class="callout-content dark:text-neutral-300"> <div class="callout-title font-semibold mb-1">Custom Warning Title<br></div> <div class="callout-body"><p>This warning has a custom title instead of just “Warning”.</p></div> </div> </div> <h3 id="legacy-syntax-deprecated">Legacy Syntax (Deprecated)</h3> <p>The old shortcode syntax still works but shows a deprecation warning:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="cl">{{< callout note >}} </span></span><span class="line"><span class="cl">This still works but is deprecated. Use the Markdown syntax above! </span></span><span class="line"><span class="cl">{{< /callout >}} </span></span></code></pre></div><h3 id="css-customization">CSS Customization</h3> <p>Hugo Blox generates callouts with semantic CSS classes and data attributes, making customization easy. Each callout has:</p> <ul> <li>Base class: <code>.callout</code></li> <li>Type-specific data attribute: <code>data-callout="note"</code></li> <li>Component classes: <code>.callout-icon</code>, <code>.callout-title</code>, <code>.callout-content</code>, <code>.callout-body</code></li> </ul> <p><strong>Custom CSS Example</strong> (add to your <code>assets/css/custom.css</code>):</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-css" data-lang="css"><span class="line"><span class="cl"><span class="c">/* Customize NOTE callouts */</span> </span></span><span class="line"><span class="cl"><span class="p">.</span><span class="nc">callout</span><span class="o">[</span><span class="nt">data-callout</span><span class="o">=</span><span class="s2">"note"</span><span class="o">]</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">border-left-width</span><span class="p">:</span> <span class="mi">8</span><span class="kt">px</span><span class="p">;</span> </span></span><span class="line"><span class="cl"> <span class="k">box-shadow</span><span class="p">:</span> <span class="mi">0</span> <span class="mi">4</span><span class="kt">px</span> <span class="mi">6</span><span class="kt">px</span> <span class="mi">-1</span><span class="kt">px</span> <span class="nb">rgb</span><span class="p">(</span><span class="mi">0</span> <span class="mi">0</span> <span class="mi">0</span> <span class="o">/</span> <span class="mf">0.1</span><span class="p">);</span> </span></span><span class="line"><span class="cl"><span class="p">}</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c">/* Make SUCCESS callouts pulse */</span> </span></span><span class="line"><span class="cl"><span class="p">.</span><span class="nc">callout</span><span class="o">[</span><span class="nt">data-callout</span><span class="o">=</span><span class="s2">"success"</span><span class="o">]</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">animation</span><span class="p">:</span> <span class="n">pulse</span> <span class="mi">2</span><span class="kt">s</span> <span class="kc">infinite</span><span class="p">;</span> </span></span><span class="line"><span class="cl"><span class="p">}</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c">/* Custom icon styling */</span> </span></span><span class="line"><span class="cl"><span class="p">.</span><span class="nc">callout-icon</span> <span class="nt">svg</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">transition</span><span class="p">:</span> <span class="k">transform</span> <span class="mf">0.2</span><span class="kt">s</span><span class="p">;</span> </span></span><span class="line"><span class="cl"><span class="p">}</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="p">.</span><span class="nc">callout</span><span class="p">:</span><span class="nd">hover</span> <span class="p">.</span><span class="nc">callout-icon</span> <span class="nt">svg</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">transform</span><span class="p">:</span> <span class="nb">scale</span><span class="p">(</span><span class="mf">1.1</span><span class="p">);</span> </span></span><span class="line"><span class="cl"><span class="p">}</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c">/* Dark mode overrides */</span> </span></span><span class="line"><span class="cl"><span class="p">@</span><span class="k">media</span> <span class="o">(</span><span class="nt">prefers-color-scheme</span><span class="o">:</span> <span class="nt">dark</span><span class="o">)</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="p">.</span><span class="nc">callout</span><span class="o">[</span><span class="nt">data-callout</span><span class="o">=</span><span class="s2">"warning"</span><span class="o">]</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">background-color</span><span class="p">:</span> <span class="nb">rgb</span><span class="p">(</span><span class="mi">92</span> <span class="mi">25</span> <span class="mi">2</span><span class="p">);</span> </span></span><span class="line"><span class="cl"> <span class="k">border-color</span><span class="p">:</span> <span class="nb">rgb</span><span class="p">(</span><span class="mi">245</span> <span class="mi">158</span> <span class="mi">11</span><span class="p">);</span> </span></span><span class="line"><span class="cl"> <span class="p">}</span> </span></span><span class="line"><span class="cl"><span class="p">}</span> </span></span></code></pre></div><p>This approach matches , ensuring your styles work across platforms.</p> <h3 id="benefits-of-the-new-syntax">Benefits of the New Syntax</h3> <ul> <li><strong>Portable</strong>: Works with GitHub, Obsidian, and other Markdown editors</li> <li><strong>Standard</strong>: Uses widely-adopted Markdown callout syntax</li> <li><strong>Clean</strong>: No Hugo-specific shortcodes needed</li> <li><strong>Future-proof</strong>: Supported by the latest Hugo versions (0.132.0+)</li> <li><strong>Customizable</strong>: Semantic CSS classes and data attributes for easy styling</li> <li><strong>Complete</strong>: All 15+ Obsidian callout types supported</li> <li><strong>Multilingual</strong>: Callout titles are automatically translated based on your site’s language (and can be customized in the language packs)</li> </ul> </article> <article> <h1>Cards</h1> <p>Mon, 01 Jan 0001 00:00:00 +0000</p> <p>A Hugo extension to create cards. Cards can be shown as links or as plain text.</p> <h2 id="usage">Usage</h2> <div class="hb-cards mt-4 grid gap-4 not-prose" style="--hb-cols: 1;"> <a class="hb-card group"href="../" > <span class="hb-card-title p-4"> <svg style="height: 1em; width: 1em;" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M4.26 10.147a60 60 0 0 0-.491 6.347A48.6 48.6 0 0 1 12 20.904a48.6 48.6 0 0 1 8.232-4.41a61 61 0 0 0-.491-6.347m-15.482 0a51 51 0 0 0-2.658-.813A60 60 0 0 1 12 3.493a60 60 0 0 1 10.399 5.84q-1.345.372-2.658.814m-15.482 0A51 51 0 0 1 12 13.489a50.7 50.7 0 0 1 7.74-3.342M6.75 15a.75.75 0 1 0 0-1.5a.75.75 0 0 0 0 1.5m0 0v-3.675A55 55 0 0 1 12 8.443m-7.007 11.55A5.98 5.98 0 0 0 6.75 15.75v-1.5"/></svg>Learn Shortcodes</span></a> <a class="hb-card group"> <span class="hb-card-title p-4">A card without an icon or link</span></a> </div> <p>is rendered by:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl">{{< cards >}} </span></span><span class="line"><span class="cl"> {{< card url="../callout" title="Callout" icon="academic-cap" >}} </span></span><span class="line"><span class="cl"> {{< card url="" title="A card without an icon" >}} </span></span><span class="line"><span class="cl">{{< /cards >}} </span></span></code></pre></div><h2 id="options">Options</h2> <table> <thead> <tr> <th>Parameter</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td><code>icon</code></td> <td>Name of the icon. Defaults to </td> </tr> <tr> <td><code>title</code></td> <td>Title heading for the card.</td> </tr> <tr> <td><code>subtitle</code></td> <td>Subtitle heading (supports Markdown).</td> </tr> <tr> <td><code>url</code></td> <td>URL</td> </tr> </tbody> </table> </article> <article> <h1>Internationalization (i18n)</h1> <p>Mon, 01 Jan 0001 00:00:00 +0000</p> <p>Hugo Blox enables you to easily edit the interface text as well as translating your site into multiple languages using Hugo’s feature.</p> <p>View the full docs at </p> </article> <article> <h1>Spoilers</h1> <p>Mon, 01 Jan 0001 00:00:00 +0000</p> <p>A Hugo shortcode to toggle collapsible content.</p> <h2 id="example">Example</h2> <details class="spoiler " id="spoiler-0"> <summary class="cursor-pointer">Click to view the spoiler</summary> <div class="rounded-lg bg-neutral-50 dark:bg-neutral-800 p-2"> <p>You found me!</p> <p>Markdown is <strong>supported</strong>.</p> </div> </details> <h2 id="usage">Usage</h2> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl">{{< spoiler text="Click to view the spoiler" >}} </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">This is the content of the details. </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">Markdown is **supported**. </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">{{< /spoiler >}} </span></span></code></pre></div> </article> <article> <h1>Steps</h1> <p>Mon, 01 Jan 0001 00:00:00 +0000</p> <p>Display a series of steps, such as for a tutorial.</p> <h2 id="example">Example</h2> <div class="hb-steps"> <h3 id="step-1">Step 1</h3> <p>The first step here…</p> <h3 id="step-2">Step 2</h3> <p>The second step here…</p> <h3 id="step-3">Step 3</h3> <p>The third step here…</p> </div> <h2 id="usage">Usage</h2> <p>Use the Markdown level-3 headings to represent step titles within the <code>steps</code> shortcode:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl">{{% steps %}} </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">### Step 1 </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">The first step here... </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">### Step 2 </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">The second step here... </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">### Step 3 </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">The third step here... </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">{{% /steps %}} </span></span></code></pre></div> </article> </main></body></html>