Roadmap: Pricing, Launch, Scaling & New Features

Raw Text

NovelAI · Follow

9 min read · Jun 2, 2021

--

5

Listen

Share

This blogpost is separated into 3 parts:

Pricing & Launch timing

Priority System

Feature Roadmap

For our first roadmap post, we are going to share some insights on the scaling & priority system behind our planned subscription pricing tiers.

Please keep in mind that this system will be experimental , and we will utilize our beta to determine if it will be sustainable over a longer period of time.

The most important question on everyone’s mind right now is the pricing , so let’s focus on that first! Our pricing tier structure is as follows and keeps future service & model expansion in mind.

What does the context size mean for you

Context size is the length the language model can remember at once. So if your context size is 700 tokens, which roughly translates to 2800 characters it means after that 2800 characters, the AI will forget what happened before that. AID had a 2800 character context length.

For NovelAI, our Tablet plan provides a 1024 token context length, and the higher plans provide a 2048 token context length, which is almost 3x of the context length AI Dungeon offers.

Even with our base plan , you still get a larger context length with NovelAI compared to AID.

*In the English language, every GPT token is 4 characters on average.

For the Tablet and Scroll plans, you are getting 4000 actions in max priority per month in total and 4000 actions in normal priority per month. After you hit 0 priority which means using 8000 actions you can still use the service but we can’t guarantee a decent speed depending on our load.

If you subscribe to Opus tier, your priority level will always be 10, so you are always prioritized on the queue regardless of how many actions you have used this week.

Launch Timing & Crowdfunding

We will launch our beta in the upcoming weeks . When it launches you will be able to join by going to our site and subscribing to our service. We will notify you from our social media accounts before we launch the service.

You can support NovelAI outside of subscribing to the beta by becoming a Patron .

Donations will not give you access to the beta but highly appreciated if you want to support our project.

We would like to make it clear that the purpose of this Patreon is to fund the startup and we intend to rely fully on subscription revenue to cover our operating expenses.

Our patreon : https://patreon.com/novelai

Insights into our GPU scaler

We briefly talked about our GPU scaler in our last blog post, The First Month of NovelAI , and we’re going to elaborate on our plan for it since our priority system heavily depends upon this scaler

Our dynamic GPU scaler is making it possible for us to get more GPUs whenever we need them. It also drops unnecessary GPU allocations when there are periods of lower demand. This allows us to automatically optimize the pricing since we will acquire the cheapest GPU per TFLOP by utilizing many GPU providers.

This is where we started thinking about an alternative to the energy system that AI Dungeon used. Since we manage our own GPUs, there are plenty of opportunities to optimize or create a better system. As you know, AI Dungeon uses a hard-capped energy limit where you can’t generate any more text if you run out of energy. In our case, as long as we have available space on a GPU, we can just let people use it even if they ran out of actions.

Processing generation requests

Every GPU node is fed by our queue that stores the generation requests. These requests are then selected according to their priority level. High-priority tier generations take precedence and get processed first.

Priority system

Our priority system is designed to be very flexible and works hand in hand with our scaler. Regardless of your subscription tier, your Priority Level will start at 10, and you are provided with 1000 actions per week at that priority level: Max Priority Actions.

Once you run out of Max Priority Actions, you will lose 1 priority level per 100 actions. After 1000 actions in normal priority mode, your priority level will be 0.

For the Tablet and Scroll plans, you are getting 4000 actions in max priority per month in total and 4000 actions in normal priority per month.

If you subscribe to Opus tier, your priority level will always be 10, so you are always prioritized on the queue regardless of how many actions you have used this week.

GPU scaling policy

As we pointed out earlier, this priority system works hand to hand with our GPU scaler. As user requests increase on our existing GPUs, we gradually rent more to fulfill them .

Your priority affects this as a coefficient, so if you have a lower priority, your load on the GPUs will affect the scaler’s decision to scale up less, which means you will have to wait in the line of the already running GPUs.

When does my priority reset?

Our system is operating on a combination of a weekly and a monthly basis. At the start of the first week, you get 1000 of Max Priority Actions. After you use them all up, your priority level starts to decrease.

After 7 days, your priority resets, and a fifth of your remaining Max Priority Actions carry over additionally to the 1000 actions you get the next week. (If you had 1000 left, you keep 200.) Which means you end up with having 1200 Max Priority actions the next week.

This weekly limit is meant to mitigate usage spikes, and we think it’s a better system than waiting for energy to recharge gradually. On the date of your subscription renewal, your max action amount resets to its original value, with no carryover Max Priority Actions from the previous month .

What does this mean for you?

As this system is very flexible, you will start losing priority slowly when you hit your action limit and never get rejected by the system. We wanted to design a system that could use our infrastructure more efficiently and maximize our total GPU throughput and leverage our custom backend. What you get out of this are more actions to use and being able to continue using the service even if you have low or 0 priority.

Context limit on plans

GPT-Neo supports a 2048 token maximum for the input context. However , making full use of this higher token context impacts our expenses . As such, we determined that a 15$ tier will allow us to provide subscribers more memory context for their stories while keeping the service viable.

Normal priority actions

Normal Priority Actions have slowly decreasing priority. Every 100 Normal Priority Actions you use in a week will decrease your priority level by 1.

If you subscribe to Opus tier, your priority level will always be 10, so you are always prioritized on the queue regardless of how many actions you have used this week.

Conclusion on priority

We have developed this system to provide you with a better user experience. This should allow us to provide a smoother, stabler play experience to our users.

Again, this is an experimental system that we believe will work better than AID’s energy, and we will test this system extensively during the beta to adjust pricing and subscription privileges.

We will be gathering and considering your feedback on our system and its pricing on all of our social channels, such as Discord , Reddit , and the comments on this Medium article.

Roadmap

Experimental AI features

Within the near future (Soon™) we plan to implement a robust inventory system for NovelAI playthroughs.

Particularly, received items will be procedurally generated and be cohesive within the context at hand.

Imagine a scenario where you receive a scythe on a farm. During your time on the farm the use case of the scythe is to aid in the tending of crops. The description and possible uses of this item will reflect that and be enforced by our dynamic inventory system.

As you enter a new scene, such as the lair of a big mean dragon, your character realizes that the scythe is actually the perfect weapon for fighting the dragon. At this point our dynamic inventory system will generate natural language attributes and descriptions for the scythe: while reflecting its origin as a farming tool, it also allows for the possibility of using it as a weapon.

These descriptions and knowledge bases are going to be persistent — entirely independent of context length. Our knowledge bases will exist outside of the context and as such do not consume precious token space.

As you continue the story, these descriptions might change but the game will never forget that the scythe was originally obtained at the aforementioned farm. If you ever meet the farmer again, perhaps they’ll ask for the scythe back or for a retelling of the adventures it’s seen.

Website

Before the Beta

We’ve been hard at work since the Alpha ended and we’d like to show you some of the things we’ve been working on based on the feedback we received.

Theme Editor

Most of you already saw the theme editor in our previous blog, but what you didn’t see is that we now also have font options. Currently, there are four options for header and paragraph fonts and we plan to add a few more as we go through the beta.

More settings

We’ve also added several new requested settings: It’s now possible to enable or disable the input box, output box highlighting for retry and undo, as well as browser spell checking.

After evaluating the feedback on our UI elements, we realized that the setting toggles were not clear and left too much room for misunderating. Thus, we updated the design of these elements to be much clearer.

Undo and Retry Highlighting

Many users found it hard to see which text the retry and undo buttons would affect. Now, any text that would be removed on retry or undo is highlighted when your mouse hovers over the button.

Tutorial Messages & Better Tooltips

For an easier start, we added small messages that guide you into the functionality of the editor. We also improved the tooltips on various elements like the AI generation settings.

Hotkeys

While not fully done yet, we added hotkey support to control various elements of the interface. We’re currently working out which hotkeys would be the best for the application and hope to have an intuitive set ready when the beta starts.

Planned features during the Beta

Lore Book

A much requested feature is the enrichment of the context with additional context dependent information. We will start with a simple word-matching version of this world information feature, and will expand it with more intelligent matching in the future.

Import of Story Exports from other Storytellers

Acknowledging many users’ requests, we are working on an option to import exported stories from AI Dungeon or KoboldAI. This will even include world info!

Scripting

Extensive scripting support is planned to be added over the course of the beta. More details on this soon!

More Export Formats

While you can currently export stories in their native format for easy sharing including their history, settings and more, we will add exports for other formats as well: As plain text, as a prompt, or even as a PDF.

Read Mode

Better Mobile Support

While NovelAI works well on mobile device browsers already, we will continue to enhance the mobile experience as much as possible.

It is also available as a progressive web application, both on mobile and desktop.

Custom Context Menus

We will work on a custom context menu for lots of functionalities — like adding lore book entries by selecting the text, right clicking it, and selecting add. This and other features will simplify navigating the UI.

We will be gathering and considering your feedback on these features and the pricing on all of our social channels, such as Discord , Reddit , and the comments on this Medium article.

Our social media:

Discord: https://discord.gg/novelai

Twitter: https://twitter.com/novelaiofficial

Reddit: https://www.reddit.com/r/NovelAi/

Website: https://novelai.net/

Patreon: https://patreon.com/novelai

Single Line Text

NovelAI · Follow. 9 min read · Jun 2, 2021. -- 5. Listen. Share. This blogpost is separated into 3 parts: Pricing & Launch timing. Priority System. Feature Roadmap. For our first roadmap post, we are going to share some insights on the scaling & priority system behind our planned subscription pricing tiers. Please keep in mind that this system will be experimental , and we will utilize our beta to determine if it will be sustainable over a longer period of time. The most important question on everyone’s mind right now is the pricing , so let’s focus on that first! Our pricing tier structure is as follows and keeps future service & model expansion in mind. What does the context size mean for you. Context size is the length the language model can remember at once. So if your context size is 700 tokens, which roughly translates to 2800 characters it means after that 2800 characters, the AI will forget what happened before that. AID had a 2800 character context length. For NovelAI, our Tablet plan provides a 1024 token context length, and the higher plans provide a 2048 token context length, which is almost 3x of the context length AI Dungeon offers. Even with our base plan , you still get a larger context length with NovelAI compared to AID. *In the English language, every GPT token is 4 characters on average. For the Tablet and Scroll plans, you are getting 4000 actions in max priority per month in total and 4000 actions in normal priority per month. After you hit 0 priority which means using 8000 actions you can still use the service but we can’t guarantee a decent speed depending on our load. If you subscribe to Opus tier, your priority level will always be 10, so you are always prioritized on the queue regardless of how many actions you have used this week. Launch Timing & Crowdfunding. We will launch our beta in the upcoming weeks . When it launches you will be able to join by going to our site and subscribing to our service. We will notify you from our social media accounts before we launch the service. You can support NovelAI outside of subscribing to the beta by becoming a Patron . Donations will not give you access to the beta but highly appreciated if you want to support our project. We would like to make it clear that the purpose of this Patreon is to fund the startup and we intend to rely fully on subscription revenue to cover our operating expenses. Our patreon : https://patreon.com/novelai. Insights into our GPU scaler. We briefly talked about our GPU scaler in our last blog post, The First Month of NovelAI , and we’re going to elaborate on our plan for it since our priority system heavily depends upon this scaler. Our dynamic GPU scaler is making it possible for us to get more GPUs whenever we need them. It also drops unnecessary GPU allocations when there are periods of lower demand. This allows us to automatically optimize the pricing since we will acquire the cheapest GPU per TFLOP by utilizing many GPU providers. This is where we started thinking about an alternative to the energy system that AI Dungeon used. Since we manage our own GPUs, there are plenty of opportunities to optimize or create a better system. As you know, AI Dungeon uses a hard-capped energy limit where you can’t generate any more text if you run out of energy. In our case, as long as we have available space on a GPU, we can just let people use it even if they ran out of actions. Processing generation requests. Every GPU node is fed by our queue that stores the generation requests. These requests are then selected according to their priority level. High-priority tier generations take precedence and get processed first. Priority system. Our priority system is designed to be very flexible and works hand in hand with our scaler. Regardless of your subscription tier, your Priority Level will start at 10, and you are provided with 1000 actions per week at that priority level: Max Priority Actions. Once you run out of Max Priority Actions, you will lose 1 priority level per 100 actions. After 1000 actions in normal priority mode, your priority level will be 0. For the Tablet and Scroll plans, you are getting 4000 actions in max priority per month in total and 4000 actions in normal priority per month. If you subscribe to Opus tier, your priority level will always be 10, so you are always prioritized on the queue regardless of how many actions you have used this week. GPU scaling policy. As we pointed out earlier, this priority system works hand to hand with our GPU scaler. As user requests increase on our existing GPUs, we gradually rent more to fulfill them . Your priority affects this as a coefficient, so if you have a lower priority, your load on the GPUs will affect the scaler’s decision to scale up less, which means you will have to wait in the line of the already running GPUs. When does my priority reset? Our system is operating on a combination of a weekly and a monthly basis. At the start of the first week, you get 1000 of Max Priority Actions. After you use them all up, your priority level starts to decrease. After 7 days, your priority resets, and a fifth of your remaining Max Priority Actions carry over additionally to the 1000 actions you get the next week. (If you had 1000 left, you keep 200.) Which means you end up with having 1200 Max Priority actions the next week. This weekly limit is meant to mitigate usage spikes, and we think it’s a better system than waiting for energy to recharge gradually. On the date of your subscription renewal, your max action amount resets to its original value, with no carryover Max Priority Actions from the previous month . What does this mean for you? As this system is very flexible, you will start losing priority slowly when you hit your action limit and never get rejected by the system. We wanted to design a system that could use our infrastructure more efficiently and maximize our total GPU throughput and leverage our custom backend. What you get out of this are more actions to use and being able to continue using the service even if you have low or 0 priority. Context limit on plans. GPT-Neo supports a 2048 token maximum for the input context. However , making full use of this higher token context impacts our expenses . As such, we determined that a 15$ tier will allow us to provide subscribers more memory context for their stories while keeping the service viable. Normal priority actions. Normal Priority Actions have slowly decreasing priority. Every 100 Normal Priority Actions you use in a week will decrease your priority level by 1. If you subscribe to Opus tier, your priority level will always be 10, so you are always prioritized on the queue regardless of how many actions you have used this week. Conclusion on priority. We have developed this system to provide you with a better user experience. This should allow us to provide a smoother, stabler play experience to our users. Again, this is an experimental system that we believe will work better than AID’s energy, and we will test this system extensively during the beta to adjust pricing and subscription privileges. We will be gathering and considering your feedback on our system and its pricing on all of our social channels, such as Discord , Reddit , and the comments on this Medium article. Roadmap. Experimental AI features. Within the near future (Soon™) we plan to implement a robust inventory system for NovelAI playthroughs. Particularly, received items will be procedurally generated and be cohesive within the context at hand. Imagine a scenario where you receive a scythe on a farm. During your time on the farm the use case of the scythe is to aid in the tending of crops. The description and possible uses of this item will reflect that and be enforced by our dynamic inventory system. As you enter a new scene, such as the lair of a big mean dragon, your character realizes that the scythe is actually the perfect weapon for fighting the dragon. At this point our dynamic inventory system will generate natural language attributes and descriptions for the scythe: while reflecting its origin as a farming tool, it also allows for the possibility of using it as a weapon. These descriptions and knowledge bases are going to be persistent — entirely independent of context length. Our knowledge bases will exist outside of the context and as such do not consume precious token space. As you continue the story, these descriptions might change but the game will never forget that the scythe was originally obtained at the aforementioned farm. If you ever meet the farmer again, perhaps they’ll ask for the scythe back or for a retelling of the adventures it’s seen. Website. Before the Beta. We’ve been hard at work since the Alpha ended and we’d like to show you some of the things we’ve been working on based on the feedback we received. Theme Editor. Most of you already saw the theme editor in our previous blog, but what you didn’t see is that we now also have font options. Currently, there are four options for header and paragraph fonts and we plan to add a few more as we go through the beta. More settings. We’ve also added several new requested settings: It’s now possible to enable or disable the input box, output box highlighting for retry and undo, as well as browser spell checking. After evaluating the feedback on our UI elements, we realized that the setting toggles were not clear and left too much room for misunderating. Thus, we updated the design of these elements to be much clearer. Undo and Retry Highlighting. Many users found it hard to see which text the retry and undo buttons would affect. Now, any text that would be removed on retry or undo is highlighted when your mouse hovers over the button. Tutorial Messages & Better Tooltips. For an easier start, we added small messages that guide you into the functionality of the editor. We also improved the tooltips on various elements like the AI generation settings. Hotkeys. While not fully done yet, we added hotkey support to control various elements of the interface. We’re currently working out which hotkeys would be the best for the application and hope to have an intuitive set ready when the beta starts. Planned features during the Beta. Lore Book. A much requested feature is the enrichment of the context with additional context dependent information. We will start with a simple word-matching version of this world information feature, and will expand it with more intelligent matching in the future. Import of Story Exports from other Storytellers. Acknowledging many users’ requests, we are working on an option to import exported stories from AI Dungeon or KoboldAI. This will even include world info! Scripting. Extensive scripting support is planned to be added over the course of the beta. More details on this soon! More Export Formats. While you can currently export stories in their native format for easy sharing including their history, settings and more, we will add exports for other formats as well: As plain text, as a prompt, or even as a PDF. Read Mode. Better Mobile Support. While NovelAI works well on mobile device browsers already, we will continue to enhance the mobile experience as much as possible. It is also available as a progressive web application, both on mobile and desktop. Custom Context Menus. We will work on a custom context menu for lots of functionalities — like adding lore book entries by selecting the text, right clicking it, and selecting add. This and other features will simplify navigating the UI. We will be gathering and considering your feedback on these features and the pricing on all of our social channels, such as Discord , Reddit , and the comments on this Medium article. Our social media: Discord: https://discord.gg/novelai. Twitter: https://twitter.com/novelaiofficial. Reddit: https://www.reddit.com/r/NovelAi/ Website: https://novelai.net/ Patreon: https://patreon.com/novelai.