Slider Settings

 

Config Preset

The Config Preset dropdown menu displays all of your saved and imported presets at the top, with the NovelAI defaults below. Each model comes with their own variety of default presets, with the majority tuned for creative writing, and some tuned for certain writing or generation styles. The pen icon beside the dropdown is for renaming the selected preset.

The Import and Export buttons can be used to share your preset or import others people have shared, and whenever you modify one of the settings below, the Update active Preset popup will appear below the dropdown. The Update active Preset dropdown allows you to save the current preset, Reset Changes back to the presets' original settings, or save your current changes as a new preset!

 

Generation Options

The Generation Options section of the page contains the three baseline generation settings: Randomness, Output Length, and Repetition Penalty. These settings are some of the clearest and easiest to adjust on the fly, and for the most part can be changed without having to adjust any of the Samplers below.

  • Randomness

    The AI model gives probabilities for tokens, but doesn't choose tokens directly. When generating text, we use Probabilities to pick a token. Randomness adjusts the chance that a token is chosen. More likely tokens tend to be more relevant or 'correct', but if only the most likely tokens are picked, the generated text can become repetitive and stale. Settings such as *Randomness and the Samplers below help balance this by creating a more or less diverse pool of tokens to be chosen from, depending on how they are used.

    Randomness = 1 means token probabilities follow the typical distribution of text. It is the default setting, and the recommended choice if you are inexperienced with the usage of Randomness.

    Randomness < 1 means likely tokens become even likelier and unlikely tokens become rarer. Low Randomness creates higher consistency of logical tokens, but its downside is repetitiveness and lowered creativity. When Randomness is too low, the generated text can become stuck in repetition, which is undesirable. This is an oft-discussed topic in machine learning—that picking only high-probability choices for each token creates bad output over the long run.

    Randomness > 1 means the probability of generating every token becomes more equal. Higher Randomness allows more creativity, but its downside is introducing increased mistakes, such as logical errors and typos, in your output.

    Keeping Randomness near 1 is the recommended practice. If Randomness is higher than 1, it is helpful to use a sampler to help delete low-probability tokens.

    For example, if we use the above image as our prompt and view the probabilities of the tokens to be generated after was...

    Randomness 1.0 Randomness 1.25

    This page shows the highest-probability tokens. In the right-hand column, 1.25 Randomness has lowered the after probabilities of the tokens, with the highest tokens decreasing more. This is because high Randomness gives every token a more equal chance at being generated, and only the high probability tokens are listed on the page. The low-probability tokens, not on the page, have their after probabilities raised. High Randomness makes tokens more equal in their probabilities, but it never changes their order.

  • Output Length

    The Output Length setting controls the maximum amount of text characters that the AI can generate per output, with a minimum of 4 and a maximum of 600 characters depending on your Subscription Tier. Be aware that longer output lengths can vary in quality due to the nature of the AI's generation, while shorter ones will tend to stay on topic better.

  • Repetition Penalty

    The Repetition Penalty slider applies a penalty to the probability of tokens that appear in context, with those that appear multiple times being penalized more harshly. Higher values apply a harsher penalty, so setting this slider too high can result in output degradation as the AI runs out of words to use, while too low of a setting can cause the AI to continuously repeat words or punctuation. For example, if you wanted the AI to mention specific character names or details more often, you would lower this slider. You would increase it if you wanted it to use more varied word choices.

 

Advanced Options

Sampling

There is a tradeoff in text generation; high-probability tokens are good, but you need to sample a diversity of them, while low-probability tokens are frequently junk. The sampling options below are different ways to delete low-probability tokens. This achieves something that Randomness cannot: quashing the low-probability tokens (improving quality), without unduly increasing the high-probability tokens (which would decrease diversity). Our recommended choices for a main sampler are Tail-Free, Typical, and Nucleus.

  • Mirostat

    Mirostat has two sliders, Tau and Learning Rate. This sampler attempts to keep outputted tokens at a given stochasticity specified by the Tau value, with higher settings potentially leading to more creative output. The Learning Rate slider specifies how quickly Mirostat changes its estimate of stochasticity in relation to context, with a setting of 1 being near instantaneous, easing up as the setting is lowered. Combining this sampler with others in the Settings Order is not recommended.

  • Nucleus

    Nucleus sampling, also known as Top-P, sorts the tokens from highest to lowest probability, then deletes the low probability tokens. The threshold for deletion is to keep enough tokens that the sum of their probability is equal to the slider's value. Nucleus sampling increases output consistency, but lower probability tokens will be lost in the process, sacrificing creativity. Small adjustments are recommended when experimenting, as lower settings on this slider remove more tokens.

  • Tail-Free

    Intended to replace Top K and Nucleus sampling, Tail-Free uses a mathematical formula to calculate the 'tail' of an outputs' probabilities. Tail, and the Tail-Free sampler are explained in detail in this blog post. It determines a threshold of the lowest probability tokens to be the 'tail' of the output's probability spread, then removes them. After this removal, the surviving tokens have their probabilities readjusted to compensate.

    To simplify, this setting helps trim some of what the formula considers the worst possible tokens from the bottom of your output's Logical Probabilities. Small adjustments are recommended when changing this slider, as the closer to 0 you set Tail-Free, the more intense the deletion threshold becomes.

  • Top A

    Top-A sampling deletes every token with probability less than (maximum token probability)^2 * A. Essentially, tokens are deleted if their probability is much smaller than the top token's probability. Higher Top-A values are stricter, cutting more tokens.

  • Top K

    Top-K is the simplest sampler: the number of tokens it considers is equal to your slider setting. For example, if you set Top-K to 10, the sampler will keep the 10 most likely tokens, and remove the rest.

Goose Tip: Setting Top-K to 1 ensures you get the same token every time when retrying generations! The text quality won't be high, but it can be useful for testing.

  • Typical

    Typical sampling is one of the more complex options available. On each output token generated, it estimates "the expected probability of a token" using an entropy calculation. If a token's probability is too high or too low compared to this expected probability, it is deleted. The Typical setting decides the proportion of tokens to keep. 1 keeps all tokens, and 0 deletes all tokens. Note that this sampler deletes high-probability tokens, which is very unusual. It will generate varied and diverse output, but of lower quality.

  • Change Settings Order

The Order Settings window allows you to change the order of samplers, which are applied from top to bottom. Use the arrow buttons or drag each box individually to rearrange the sampling order, or toggle them with the buttons on the right. Temperature (Randomness) cannot be disabled.

The order in which you apply samplers can have unexpected and unpredictable effects. Consider starting with a default config preset and experimenting.

 

Repetition Penalty

The options in the Repetition penalties section, as well as the Alternative Repetition Penalty section below, are all intended to make your generations less repetitive.

  • Phrase Repetition Penalty

    See the Advanced: Phrase Repetition Penalty page for more details.

  • Use Default Whitelist

    See the Repetition Penalty Whitelist page for a full list of whitelisted tokens.

  • Range

    Repetition Penalty Range is how many tokens, starting from the bottom of your Story Context, will have Repetition Penalty settings applied. When set to the minimum of 0 (off), repetition penalties are applied to the full range of your output, which is the same as having the slider set to the maximum of your Subscription Tier. This slider only functions when Dynamic Range is disabled.

  • Slope

    The Slope slider dictates what percentage of your set Repetition Penalties (all except for Phrase Repetition Penalty) are applied to tokens in context with respect to their distance from the most recent token in context. When disabled, no sloping is applied, and all penalties apply themselves as normal.

    When the Slope is set to a value below or equal to 1, only the final token receives 100% of the penalty values, with prior tokens experiencing a reduction in the penalty percentage which gets smoother and more gradual the closer your set Slope value is to 0. When set to 1 exactly, this reduction in percentage is an identical amount for each token that stacks, making your slope a straight upward line.

    At values above 1, the Slope changes from a straight line to a stair-step shape, becoming more intense the closer your Slope value gets to the maximum of 10. In this range, it is possible for multiple of the most recent tokens to receive 100% of your penalty values, however a "cliff edge" forms where prior tokens then suddenly have the penalty percentage reduced drastically. At a Slope value of 10, half of context recieves 100% of your penalty values, while the other half recieves no penalties at all.

  • Dynamic Range

    When enabled, the Dynamic Range toggle makes it so Repetition Penalty settings are only applied to Story text. This means that penalties are not applied to Memory, Author's Note, or Lorebook text within . Enabling this can allow the AI to mention lore or descriptions mentioned in those sections more often, and prevents the Range slider from being adjusted.

 

Alternative Repetition Penalty

The settings in the Alternative Repetition Penalty section are highly advanced features. Even small adjustments made to these sliders can have vast effects on your AI generations, often cutting out too many tokens and resulting in gibberish outputs. Use very small adjustments when experimenting with these, and be wary of your Range settings when doing so.

  • Presence

    Presence penalty functions similarly to the default Repetition Penalty slider, but only applies a flat penalty each time a token appears, rather than adjusting for how often they do. Very small adjustments are recommended when experimenting with Presence penalty, as setting it too high can result in punctuation tokens quickly being penalized out of generating.

  • Frequency

    Frequency penalty applies based on how frequently a token appears, penalizing more common tokens while easing up on less common ones. If set too high, Frequency can quickly degrade outputs, so very small adjustments should be made when experimenting.