Proposal: Bits as dynamic tokens.

A proposal for starting to introduce Bits (dynamic tokens/placeholders) into WordPress.

When Blocks came into the scene Shortcodes were largely abandoned, but Shortcodes had value. They had many problems, but they also had value. It wasn’t clear at the time how to bring them back without bringing back many of the problems they brought with them, namely issues surrounding ambiguity in parsing, nesting, changing the page in dramatic ways, and providing usable content in the absence of a required pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party or theme.

Around two years ago a discussion was started for introducing dynamic tokens in the editor as placeholders for externally-sourced content. The idea was raised before the discussion was started: many developers were starting to introduce unique code in multiple blocks and plugins that looked for ways to find and replace content in the editor:

• Set an image URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org to the post’s featured imageFeatured image A featured image is the main image used on your blog archive page and is pulled when the post or page is shared on social media. The image can be used to display in widget areas on your site or in a summary list of posts..
• Insert a placeholder for a subscriber’s name in an email form.
• Add the post author’s display name in a query loopLoop The Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop. template.
• Write a translatable string in a theme template inside the Site Editor.

When the HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. started developing it changed the game for these kinds of dynamic tokens. Previously the discussion was largely blocked by finding a syntax that would be reasonable for someone to type in directly, but also avoid causing all sorts of breakage to the surrounding HTML. Now, these discussions are less relevant because the HTML API provides a way to find various kinds of placeholders and then ensure a context-aware replacement and escaping when replacing them.

It provides a way for WordPress to make heuristics-based decisions on what content to allow and not to allow on output. It ensures that the output of one of the tokens doesn’t bleed into or break the page around it.

After many explorations, one form of placeholders stands out above all the others – a quirk in the HTML specification referred to within WordPress as a “funky comment.” These look like closing tags, except the tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) name is invalidinvalid A resolution on the bug tracker (and generally common in software development, sometimes also notabug) that indicates the ticket is not a bug, is a support request, or is generally invalid.. When a browser sees them they interpret them as HTML comments, removing them by default from the page (in the case that the server fails to replace them), and it’s impossible to nest them.

These funky comments are the perfect vehicle for safe fallback, human-typability, and the ability to parse and replace. While funky comments can appear in many forms, this proposal is discussing the specific form that can be used for dynamic tokens and placeholders: these are called Bits. While blocks represent rich content types, Bits represent small semantic bits of knowledge. Bits can appear in any blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. or in any HTML without requiring any changes to any existing Block code; Bits can appear anywhere.

What is a Bit?

<//wp:post-meta key="isbn">

A Bit is a small token of semantic meaning. It references content sourced beyond the post or content in which it’s found. It could refer to metadata about a post, a post metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress. field, a stock price sourced from an API call, a countdown to a particular date, the local time of a given timestamp in the reader’s timezone, a plugin URL, a view counter, a render for a math formula, or any other bit of knowledge that is provided by the server when rendering a post.

Bits are a form of horizontal composition. Blocks don’t need to know about Bits for someone to use Bits. Bits are a form of user control: anyone can add a Bit into their post without needing a developer to adjust their Blocks or theme first.

Bits are like blocks in that they comprise a name and a set of attributes, but unlike Blocks, Bits cannot nest. They are the inline analogue to block-oriented Blocks. Bits can provide some HTML, but not much. Bits are configurable by their attributes: a post date can be configured to display as “May 22” or “2010/05/22.”

Bits are registered both in PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher and also inside the editor. There’s an inspector panel for configuring a Bit based on the registration with semantic-specific controls, but Bits can also be manually typed from the visual view of the editor, and it will recognize them once typed.

Bits can be found in various contents, including plaintext and markup contexts. Bit implementations must provide both of these outputs, as well as a fallback so that they can provide some meaningful value when the necessary rendering code is missing. For example, a Bit providing a URL will might URL-encode it for URL attributes, but leave non-ASCII unicode characters in place for display purposes; a post date might return a standardized string timestamp for plaintext context but a <span>-annotated human-readable date for better CSSCSS Cascading Style Sheets. styling in markup contexts.

<!-- stored in a post -->
<time datetime="<//wp-bit:post-date format='RFC9557'>"><//wp-bit:post-date format="human-diff"></time>

<!-- rendered to the reader -->
<time datetime="2024-05-22T12:00:00+00:00">eighteen days ago</time>

Bits are parsed just like any other HTML closing tag except: the “tag name” cannot start with a-z; they extend from the start of the “tag name” until the very first > (even if it’s inside a quoted value). The attributes are parsed just like HTML attributes, meaning that there can be unquoted, single-quoted, and double-quoted attributes. The only caveat is that when found inside an HTML attribute, the quoting cannot conflictconflict A conflict occurs when a patch changes code that was modified after the patch was created. These patches are considered stale, and will require a refresh of the changes before it can be applied, or the conflicts will need to be resolved..

How do humans interact with Bits?

The editor has two inherent view modes for Bits: a preview mode, and a token mode. The Preview mode may show a preview of the replaced value of the Bit where it’s found, and it may indicate that the Bit is there through some visual indication or otherwise. The token mode shows the Bits as placeholders indicating which kind of Bit they are.

Bits are designed so that someone who is used to working with them can enter them directly as text, but people won’t need to know anything about their syntax in order to use them. Bit registration in the editor provides a name, a description, and some additional metadata just like Blocks to make it possible to provide a discoverable system for finding and configuring them.

The Bits inserter appears when typing //. Whereas the slash inserter shows Blocks on a single /, if someone types a second, they will instead see a list of Bits that will auto-filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. as they continue to type. The // for the slash inserter corresponds to the Bit syntax <//wp-bit:core/hello-dolly>.

Bit registration in the editor also provides an optional configuration panel akin to Block Inspector controls. Since each Bit carries its own semantics, these controls guide authors into how to configure Bit, maybe by selecting formatting options, choosing an associated post ID (by searching for its title), or choosing which of several options to enable.

What about Block Bindings?

Bits and Block Bindings are related but complementary systems. While Block Bindings can be thought of primarily as a developer-oriented API, where a developer can open up a given block or a subset of a block’s attributes to be replaced by some other source of data, Bits are primarily author-oriented, giving end-users the ability to add sourced content anywhere.

There is likely a large overlap in the kinds of data sources that power each system. Ideally, the registered sources will be compatible with both.

Proposal

The HTML API already introduced the concept of a “funky comment,” which is the tag closer with an invalid tag name. For WordPress 6.6 this document is only proposing to unlock storing the funky comments in CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. so that GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ the plugin can experiment with various prototypes of the Bit system with its full lifecycle. Currently this requires combining a Gutenberg patchpatch A special text file that describes changes to code, by identifying the files and lines which are added, removed, and altered. It may also be referred to as a diff. A patch can be applied to a codebase for testing. and a Core patch.

The only thing required for WordPress 6.6 is a big-fix to existing code, which would be useful even if Bits don’t come to be, and even if they use a different syntax: WordPress attempts to separate HTML comments from other tags, but it’s unaware of the myriad ways that invalid HTML turns into comments. Core-61009 introduces a patch that makes Core more aware of a couple new types of syntax-turned comments.

By opening up the ability to store Bits in the database, it makes it easier to start exploring Bits as a broader system, including what ought to be built an how. Until then, it remains cumbersome. Even in the case that Bits use a different syntax, this patch is still improves WordPress’ understanding of HTML.

---

With the ability to store Bits in the database, work should progress rapidly during the WordPress 6.7 development cycle, building up editor flows to discover, configure, and render Bits. Work will be explored in Core for registering them on the backend, and it will likely work together with a system for HTML templating powered by the HTML API.

---

For future releases and design, your feedback is invited in defining the interfaces for registering, displaying, interacting with, and governing Bits.

• Introduce Bits in Core. [#6390]
• Prior patch exploring a server registration interface. [Gutenberg#42015]
• Connect Bits in the editor inside RichText. [Gutenberg#60735]

Acknowledgements

Thanks to @annezazu, @antonvlasenko, @ironprogrammer, @ellatrix, and @gziolo (and anyone else I forgot) for review and feedback on this post before publishing.