How to converge Subresource Loading with WebBundle APIs, <script type=webbundle>, with Bundle Preloading proposal

[hayatoito]

Let's start a discussion how to converge the current Subresource Loading with WebBundles with the Bundle Preloading proposal.

Regarding their declarative syntax, Subresource loading with WebBundles is using this syntax:

<script type=webbundle>
    {
        "source": "foo.wbn",
        "resources": ["css/a.css", "css/b.css"]
    }
</script>

while the Bundle Preloading is using this syntax:

<script type="bundlepreload">
      {
          "source": "%PUBLIC_URL%/static.wbn",
          "resources": [
               "%PUBLIC_URL%/static/js/2.f2a9fc33.chunk.js",
               "%PUBLIC_URL%/static/js/3.b6022136.chunk.js",
               "%PUBLIC_URL%/static/js/runtime-main.2f0e8e04.js",
               "%PUBLIC_URL%/static/js/main.4b569d21.chunk.js",
               "%PUBLIC_URL%/static/css/main.a617e044.chunk.css",
               "%PUBLIC_URL%/static/media/logo.6ce24c58.svg"
              ]
          }
</script>

This example is taken from here.

I'm wondering how we can converge the syntax as much as possible. I know they solve slightly different goals, however, I think there are many common factors, and Bundle Preloading is listed as a target of v2 in the Subresource Loading's long term plan.

We'd appreciate any ideas or opinions what is the best way to converge both as much as possible.

Just for reference, Speculation Rules try to have a flexible syntax for defining what outgoing links can be prepared speculatively before navigation. I'm wondering how we can achieve the same goal to have a flexible syntax which is expressive enough to cover both cases

Before we ship Subresource Loading with WebBundles MVP, we'd like to make sure the syntax used there is naturally extended to cover Bundle Preloading too.

Related issues are: #648, #670, #697

[hayatoito]

w3ctag/design-reviews#616 (TAG review) might be related too.

[hayatoito]

I've found one more gap between two proposals, "how to resolve a relative URL".

In Subresource loading with WebBundles, a relative URL in resources is resolved on document's base URL.

https://github.com/WICG/webpackage/blob/main/explainers/subresource-loading.md#the-main-document

A URL in source and resources can be a relative URL to a document. A browser must parse a URL using document's base URL.

In Bundle Preloading, a relative URL specified in resources is resolved on bundle's URL.

https://github.com/WICG/bundle-preloading/blob/main/overview.md

Because of same origin and path restrictions, the resources referenced by the bundle must correspond to URLs with the same base URL as the bundle itself. If they use relative locations, those are resolved taking the location of the bundle as the base URL (see RFC7230 section 2.7 and RFC3986 section 5.1).

We might want to align somehow. I am feeling that using a bundle's URL is more intuitive.

At the era of <link>-based API, using a document's base URL might make sense because resources are specified in the link element's attribute,

Now we use <script>-based APIs, and resources is specified in JSON, so we don't have any strong reason to use document's base URL.

@irori What do you think?

[irori]

+1 to use the bundle's URL as the base URL. I think that is more intuitive.

[hayatoito]

Thanks. Let's update the explainer and our implementation too.

[jeffkaufman]

I like this as well: in experimenting with #624 we are currently injecting a <base href=https://invalid.example> so that scheme-relative URLs will be https. This is a bit of a hack, and it also does not cover cases where the ad has a same-origin iframe. This change would fix it at the browser level.

[hayatoito]

Thanks for the helpful feedback! That can be another good reason we should prefer a bundle's URL as a base URL.

Thanks. Let's update the explainer and our implementation too.

For the record, we updated at:

• Explainer PR: Resolve a relative URL in resources on a bundle's URL, instead of doc… #700
• Chromium's implementation: https://chromium-review.googlesource.com/c/chromium/src/+/3244822

[irori]

I like this as well: in experimenting with #624 we are currently injecting a <base href=https://invalid.example> so that scheme-relative URLs will be https. This is a bit of a hack, and it also does not cover cases where the ad has a same-origin iframe. This change would fix it at the browser level.

Let me clarify that #700 only changes how to resolve relative URLs in the JSON rule in <script type="webbundle">.

Relative URL resolution in iframe document still uses the document's URL (e.g. uuid-in-package:...) as the base, so it wouldn't unnecessitate the hack.

[felipeerias]

I have opened a PR at WICG/bundle-preloading#28 so the prototype server will be able to handle both absolute and relative URLs in <script type="bundlepreload"> elements.

This means that

<script type="bundlepreload">
    {
        "source": "assets/img.wbn",
        "resources": [
            "img/01.png",
            "img/02.png"
        ]
    }
</script>

will be equivalent to

<script type="bundlepreload">
    {
        "source": "http://localhost:8080/assets/img.wbn",
        "resources": [
            "http://localhost:8080/assets/img/01.png",
            "http://localhost:8080/assets/img/02.png"
        ]
    }
</script>

[hayatoito]

Just an idea, I'm wondering whether we can converge the syntax surface by introducing, such as, use_cache or something, in JSON?

For example

<script type=webbundle>
    {
        source: "foo.wbn",
        use_cache: true,
        resources: ["css/a.css", "css/b.css"]
    }
</script>

• If use_cache is true, this behaves as Bundle Preloading does.
• If use_cache is false, this behaves as Subresource Loading with Web Bundles does.
• Note that either the name of use_cache, or its default value (true or false), doesn't matter much at this point. We can find a good key name or a good default value later.

Is there any implications if we share a syntax in this way? If I miss something, please let me know that!

[felipeerias]

@hayatoito

I'm having some trouble understanding which resources would be affected by use_cache: false.

For example, with this code:

<script type="webbundle">
{
   source: "./bundle.wbn",
   resources: ["one.png", "two.png", "three.png"],
   use_cache: false
}
</script>

<img src="one.png"/>
<img src="two.png"/>

My understanding is that the browser would not fetch one.png and two.png individually; instead, it would fetch bundle.wbn and retrieve those two resources from there.

This is pretty much how it would work for Bundle Preloading as well. In our proposal, all three resources would then be stored in the regular HTTP cache by default (we are looking into using cache_name to select a specific cache, like in your proposal).

With use_cache: false, which resources would end up in the cache? None of them? All except three.png?

[felipeerias]

I have also been looking at errors and edge cases. These are some of them, with our interpretation of what would be the right behaviour for each:

• client is not able to request a bundle
  • individual resources are fetched one by one as needed by the document, receiving the same data as if they were requested in a bundle
• bundle includes an error response (e.g. 404) for a resource
  • client does not need to request the resource individually again
  • what happens when the client requests a resource that exists but is outside the scope of the bundle? should we return a specific error code for this case?
• bundle does not include any response for a requested resource
  • this is an error on the part of the server
  • what should the client do?
• server returns more resources than what was initially requested
  • could happen with a server that does not understand subsetting and simply returns the whole bundle file
  • we could either discard those extra resources, or store them but only make them available after they have been requested in another preload tag

[hayatoito]

Thanks for the feedback!

With use_cache: false, which resources would end up in the cache? None of them? All except three.png?

The current behavior of "Subresource Loading" (which is use_cache: false here) is:
A browser doesn't use a HTTP cache at all for an individual subresource. No read, no store from/to HTTP cache.
A HTTP cache is only used for a bundle itself.

[felipeerias]

@hayatoito

Thank you very much for the explanation.

In the example above this means that only bundle.wbn would be stored in cache, right?

With Bundle Preloading, only individual resources would be stored in cache. We do this because the bundle that the client receives (not the one stored in the server) will probably change a lot, because its contents depend on the resources listed on each tag and on the state of the cache when the preloading request is made.

What is the use case that you are addressing with use_cache: false?

[hayatoito]

Thanks!

How to cache for each sub resources in each webbundle has been a tricky issue for us. As far as I remember, several approaches have been proposed. e.g. Dynamic bundle serving with WebBundles, ResourceBundles, BundlePreloading, and maybe more coming?

That's a reason we can't have it in V1, and explore this area in v2 once we get more insights and feedback in real use cases.

[hayatoito]

Update:

Chromium is doing an origin trial for Subresource Loading with Web Bundles.

Given that we've got a positive feedback from OT participants, the next important step is to update several Web standards, such as <script> in HTML Standards, Fetch and etc, as a preparation for shipping MVP (aka v1 #648).

If someone finds any concerns about the current <script>-based syntax and its semantics which may be an issue for our long term plan (v2, v3), we'd appreciate it if you can share it!

We want to find any potential compatibility issues between the current one (v1) and future ones (v2, v3,...), as early as possible, before trying to update relevant Web standards so that we don't regret anything.

• The draft one for <script>

[hayatoito]

[Issue Triage]
Let me close this tentatively.