Blueprints: Add setSiteLanguage step to change the language #1538

bgrgicak · 2024-06-24T07:51:19Z

Motivation for the change, related issues

To unlock Playground to non-English speakers we want to make it easier for sites to be loaded in a local language.

Users can easily switch the site language by adding a language=LANGUAGE_CODE query parameter.

Implementation details

This PR adds a language parameter to the Query API which adds a setSiteLanguage blueprint step.

We could have achieved this without a new step, but it would require adding two steps to the blueprint generator (add constant, run PHP).

The step downloads core, plugin, and theme translations from Download.WordPress.org and moves the files into the translation directory.

Step usage example:

{
	"steps": [
		{
			"step": "setSiteLanguage",
			"language": "pl_PL"
		}
	]
}

Testing Instructions (or ideally a Blueprint)

Checkout this branch
Open this Playground url
Confirm that the theme is translated (the menu label should be Menú)
Check the admin bar and confirm it's translated
Open Akismet settings (/wp-admin/options-general.php?page=akismet-key-config&view=start) and confirm they are translated

bgrgicak · 2024-06-24T08:39:43Z

@adamziel do you know why the documentation build is failing? I tried debugging it locally but didn't get far.

packages/playground/blueprints/src/lib/steps/set-site-language.ts

akirk · 2024-06-25T11:03:11Z

I'd like to add my https://github.com/akirk/playground-step-library/blob/main/steps/setLanguage.js implementation to the discussion here. This is quite imperfect as it should use language packs (see https://api.wordpress.org/translations/core/1.0/ and https://api.wordpress.org/translations/plugins/1.0/?slug=friends) so that we get all the JSON files as well.

But note how we also need to take into account plugins and themes that are installed, we need to get their translations, too.

bgrgicak · 2024-06-26T09:12:19Z

I'd like to add my https://github.com/akirk/playground-step-library/blob/main/steps/setLanguage.js implementation to the discussion here. This is quite imperfect as it should use language packs (see https://api.wordpress.org/translations/core/1.0/ and https://api.wordpress.org/translations/plugins/1.0/?slug=friends) so that we get all the JSON files as well.

Thank you @akirk this is amazing!
I will come back to this and update the code, but let me know in case you would like to take over.

akirk · 2024-06-26T09:22:03Z

I am not sure I should, I don't know the code base well enough and will be off next week and I don't want to stall this.

I just tried to say that this is possible with the step library but that you also need to iterate the plugins to be installed, something that I think you had not addressed yet.

brandonpayton · 2024-06-27T16:48:26Z

@adamziel, @bgrgicak, and @akirk, I think this is a good example of a place that calls for separate config languages/concepts (DSLs, not human language translations):

Platform/Foundation
Actions that follow Platform-level "truth" (most of today's Blueprints)

IMO, human language selection is part of platform truth -- "The current language is X" -- as is FS construction of various mount configurations. Things like mounts can have an order of application, but they are more like platform declarations than actions. They are foundational. They are the basis upon which everything else is built.

Actions are things done upon the established foundation or platform at runtime: Install this, write that, login, etc.

There is a bit of gray area in this conceptualization of Platform vs Runtime, because each runtime action builds upon the version of reality established by the previous action. But I think we can also separate the two concepts by asking:

Is this part of setting up WordPress Playground?

Both language and directory mounts are relevant to the Playground boot process. We want to establish the filesystem before booting, and as part of booting Wordpress, we could download the language for WordPress itself.

Then, at runtime, the installPlugin step could recognize the current language and download the right translation as part of its install work.

brandonpayton · 2024-06-27T16:54:15Z

Instead of saying "separate DSLs", we could instead say things like:

Language selection belongs outside the "steps" list in the Blueprints schema
Directory mounts belong outside the "steps" list in the Blueprints schema

I also meant to ask:
What do you think?

brandonpayton · 2024-06-27T18:41:41Z

Related to the platform vs actions idea, one question is:
Does the order in which the step is included matter? Does it need to come before or after other steps?

bgrgicak · 2024-06-28T06:03:56Z

Does the order in which the step is included matter? Does it need to come before or after other steps?

If we want to translate plugin and themes, it should come at the end.

bgrgicak · 2024-06-28T06:04:27Z

Sorry I'm focused on wrapping up offline support and will get back to this next week.

bgrgicak · 2024-07-03T05:18:41Z

Language selection belongs outside the "steps" list in the Blueprints schema
Directory mounts belong outside the "steps" list in the Blueprints schema

I agree, but steps are the only thing we have today. We discussed in the past that there could be a way to separate boot actions from runtime actions in the blueprint. These look like good examples of it.
By separating them, we could also run only the runtime part of the blueprint on load and skip the boot part if Playground was already booted (for example reload when using browser storage).

bgrgicak · 2024-07-08T07:27:37Z

I moved the code to compile and implemented downloads in JS instead of PHP.

I had to use downloads.wordpress.org because translate.wordpress.org doesn't support CORS when requests come from localhost. My problem with downloads.wordpress.org is that it requires the plugin/theme version number in the URL.

@akirk @pkevan Do you know of any methods that would allow us to download the latest translations of a plugin/theme without knowing the version number?
If not we could add support to the downloads.wordpress.org API or allow CORS requests to translate.wordpress.org from localhost.

Theme/plugin translation code diff

diff --git a/packages/playground/blueprints/src/lib/steps/set-site-language.ts b/packages/playground/blueprints/src/lib/steps/set-site-language.ts
index 885c5a3d..f0de0e55 100644
--- a/packages/playground/blueprints/src/lib/steps/set-site-language.ts
+++ b/packages/playground/blueprints/src/lib/steps/set-site-language.ts
@@ -1,6 +1,12 @@
-import { StepDefinition, StepHandler } from '.';
+import {
+	InstallPluginStep,
+	InstallThemeStep,
+	StepDefinition,
+	StepHandler,
+} from '.';
 import { Blueprint } from '../blueprint';
 import { getWordPressVersion } from '@wp-playground/wordpress-builds';
+import { CorePluginReference, CoreThemeReference } from '../resources';
 
 export interface SetSiteLanguageStep {
 	step: 'setSiteLanguage';
@@ -66,10 +72,88 @@ export const compileSetSiteLanguageSteps = (
 		},
 	];
 
+	const plugins: string[] = [];
+	const themes: string[] = [];
+	for (const step of blueprint.steps) {
+		if (!step || typeof step === 'string') {
+			continue;
+		}
+		if (step.step === 'installPlugin') {
+			const pluginStep = step as InstallPluginStep<CorePluginReference>;
+			if (!pluginStep.pluginZipFile.slug) {
+				continue;
+			}
+			plugins.push(pluginStep.pluginZipFile.slug);
+		} else if (step.step === 'installTheme') {
+			const themeStep = step as InstallThemeStep<CoreThemeReference>;
+			if (!themeStep.themeZipFile.slug) {
+				continue;
+			}
+		}
+	}
+	if (plugins.length) {
+		siteLanguageSteps.push({
+			step: 'mkdir',
+			path: '/wordpress/wp-content/languages/plugins',
+		});
+	}
+	if (themes.length) {
+		siteLanguageSteps.push({
+			step: 'mkdir',
+			path: '/wordpress/wp-content/languages/themes',
+		});
+	}
+
+	for (const plugin of plugins) {
+		siteLanguageSteps.push(
+			{
+				step: 'unzip',
+				extractToPath: `/wordpress/wp-content/languages/plugins/${plugin}`,
+				zipFile: {
+					resource: 'url',
+					caption: `Downloading ${plugin}.mo`,
+					url: `https://downloads.wordpress.org/translation/plugins/${plugin}/stable/${language}.zip`, // TODO find working url
+				},
+			},
+			{
+				step: 'mv',
+				fromPath: `/wordpress/wp-content/languages/plugins/${plugin}/${plugin}-${language}.mo`,
+				toPath: `/wordpress/wp-content/languages/plugins/${plugin}-${language}.mo`,
+			},
+			{
+				step: 'rmdir',
+				path: `/wordpress/wp-content/languages/plugins/${plugin}`,
+			}
+		);
+	}
+
+	for (const theme of themes) {
+		siteLanguageSteps.push(
+			{
+				step: 'unzip',
+				extractToPath: `/wordpress/wp-content/languages/themes/${theme}`,
+				zipFile: {
+					resource: 'url',
+					caption: `Downloading ${theme}.mo`,
+					url: `https://downloads.wordpress.org/translation/themes/${theme}/stable/${language}.zip`, // TODO find working url
+				},
+			},
+			{
+				step: 'mv',
+				fromPath: `/wordpress/wp-content/languages/themes/${theme}/${theme}-${language}.mo`,
+				toPath: `/wordpress/wp-content/languages/themes/${theme}-${language}.mo`,
+			},
+			{
+				step: 'rmdir',
+				path: `/wordpress/wp-content/languages/themes/${theme}`,
+			}
+		);
+	}
+
 	blueprint.steps?.splice(setSiteLanguageStepIndex, 0, ...siteLanguageSteps);
 	return blueprint;
 };
 
 export const setSiteLanguage: StepHandler<SetSiteLanguageStep> = async () => {
-	// Implemented in packages/playground/blueprints/src/lib/compile.ts, search for `compileSetSiteLanguageSteps`
+	// Implemented in packages/playground/blueprints/src/lib/compile.ts, search for 'compileSetSiteLanguageSteps'
 };

pkevan · 2024-07-08T08:23:18Z

I moved the code to compile and implemented downloads in JS instead of PHP.

I had to use downloads.wordpress.org because translate.wordpress.org doesn't support CORS when requests come from localhost. My problem with downloads.wordpress.org is that it requires the plugin/theme version number in the URL.

@akirk @pkevan Do you know of any methods that would allow us to download the latest translations of a plugin/theme without knowing the version number? If not we could add support to the downloads.wordpress.org API or allow CORS requests to translate.wordpress.org from localhost.

Theme/plugin translation code diff

Not exactly sure what you are trying to do here, but you can use: https://api.wordpress.org/plugins/info/1.0/wordpress-beta-tester.json to grab all versions, then pick the appropriate one. Unsure if there are other methods available, or if it's worth building a specific endpoint on api.wordpress.org.

bgrgicak · 2024-07-08T08:32:12Z

Not exactly sure what you are trying to do here,

We want to add translation support to Playground.
When a user specifies a language, we automatically download all site translations.

but you can use: api.wordpress.org/plugins/info/1.0/wordpress-beta-tester.json to grab all versions, then pick the appropriate one. Unsure if there are other methods available, or if it's worth building a specific endpoint on api.wordpress.org.

Thanks! I want to avoid that because it adds one extra request for each plugin and theme we want to install.
This is why a endpoint that accepts just the plugin/theme name without a version would work well for us. It exists on https://translate.wordpress.org/projects/wp-plugins/${plugin}/dev/${lang}/default/export-translations?format=mo but we can't work with it because of CORS issues.
Would it be possible to allow CORS requests from localhost on translate.wordpress.org?

pkevan · 2024-07-08T08:43:50Z

Adding CORS support is a request via systems, see: https://make.wordpress.org/systems/?s=cors

bgrgicak · 2024-07-11T07:17:45Z

I'd like to add my akirk/playground-step-library@main/steps/setLanguage.js implementation to the discussion here. This is quite imperfect as it should use language packs (see api.wordpress.org/translations/core/1.0 and api.wordpress.org/translations/plugins/1.0?slug=friends) so that we get all the JSON files as well.

@akirk I explored this approach, but it complicates things a lot if we use blueprint steps to get translations.
If we do it with PHP it's a simple function call for core translations. With JS it's a zip download, extraction, cleanup...

When it comes to plugin and theme translations it's even more complicated because we need plugin version data and this adds extra requests.
I'm now trying to find a way to trigger the download of plugin/theme translations, but for some reason can't find how to do it. 😕

akirk · 2024-07-11T07:24:47Z

There is a function wp_download_language_pack() that should do the trick.

bgrgicak · 2024-07-11T07:26:57Z

There is a function wp_download_language_pack() that should do the trick.

That's what I'm using, but it looks like it downloads only core translations.

bgrgicak · 2024-07-11T08:28:33Z

@adamziel @akirk this should now be ready for review, sorry for the delay.

akirk · 2024-07-11T08:40:10Z

For reference, I cooked up this little piece of PHP code to install a language pack:

function install_language_pack( $type, $slug, $lang ) {
	if ( ! in_array( $type, [ 'plugin', 'theme' ] ) ) {
		return new WP_Error( 'invalid_type', __( 'Invalid type' ) );
	}
	require_once ABSPATH . 'wp-admin/includes/translation-install.php';
	require_once ABSPATH . 'wp-admin/includes/class-wp-upgrader.php';
	$language_updates = translations_api( $type . 's', compact( 'slug' ) );
	if ( is_wp_error( $language_updates ) ) {
		return $language_updates;
	}
	$packages = array_filter( $language_updates['translations'], fn( $pkg ) => $pkg['language'] === $lang );
	if ( empty( $packages ) ) {
		return;
	}
	$upgrader = new Language_Pack_Upgrader( new Automatic_Upgrader_Skin() );
	$update = (object) $packages[0];
	$update->type = $type;
	$update->slug = $slug;
	return $upgrader->bulk_upgrade( array( $update ) );
}

For example:

install_language_pack( 'plugin', 'friends', 'de_DE' );
=> array(1) {
  [0]=>
  array(7) {
    ["source"]=>
    string(70) "/Users/alex/Sites/localhost/wp/wp-content/upgrade/friends-2.9.3-de_de/"
    ["source_files"]=>
    array(7) {
      [0]=>
      string(22) "friends-de_DE.l10n.php"
      [1]=>
      string(51) "friends-de_DE-7ffd8fc6ef503cd5a5c483bb3aca9bc5.json"
      [2]=>
      string(51) "friends-de_DE-93774191f1ed3f0224434004c216505a.json"
      [3]=>
      string(16) "friends-de_DE.mo"
      [4]=>
      string(51) "friends-de_DE-9a6fba47b355b62e8f6712f5c40b6192.json"
      [5]=>
      string(51) "friends-de_DE-85be8386efdca7f7b168c9c67defc614.json"
      [6]=>
      string(16) "friends-de_DE.po"
    }
    ["destination"]=>
    string(59) "/Users/alex/Sites/localhost/wp/wp-content/languages/plugins"
    ["destination_name"]=>
    string(0) ""
    ["local_destination"]=>
    string(59) "/Users/alex/Sites/localhost/wp/wp-content/languages/plugins"
    ["remote_destination"]=>
    string(60) "/Users/alex/Sites/localhost/wp/wp-content/languages/plugins/"
    ["clear_destination"]=>
    bool(true)
  }
}

bgrgicak · 2024-07-11T10:37:33Z

For reference, I cooked up this little piece of PHP code to install a language pack:

Thanks @akirk! Let's keep the JS implementation for now, but this will be useful when we get to implementing PHP blueprints.

packages/playground/blueprints/public/blueprint-schema.json

adamziel · 2024-07-11T11:35:11Z

LGTM, I just have one note. When I use an invalid language code, like pl instead of pl_PL, I expected it to fail. However, instead of failing, it just booted an english site. Let's make sure to fail in these cases:

http://localhost:5400/website-server/?plugin=friends&theme=twentytwentythree&language=pl

akirk

I cannot say too much about whether the code approach is right (I see one unrelated change that Adam already flagged) but from a language pack perspective it looks good and works nicely.

I am happy to see that actually an old behavior on translate.wordpress.org has been improved: language packs will always be built for the newest version so that constructing the download URL using the plugin version number just works.

It used to be the case that you really needed that API call to identify the plugin version for which the last language pack was built. (for example, if it was translated to 100% in Spanish in version 1.0 and then because of a huge string update the translation ratio changed to 50% in version 2.0, the language pack would remain at 1.0, nowadays it looks like it is also build for 2.0).

packages/playground/blueprints/public/blueprint-schema.json

bgrgicak · 2024-07-12T04:42:13Z

packages/playground/blueprints/src/lib/steps/set-site-language.ts

+			/**
+			 * If a core translation wasn't found we should throw an error because it means the language is not supported or the language code isn't correct.
+			 */
+			if (type === 'core') {
+				throw new Error(
+					`Failed to download translations for WordPress. Please check if the language code ${language} is correct. You can find all available languages and translations on https://translate.wordpress.org/.`
+				);
+			}


I couldn't find a way in WP to check if the language code is valid, so I'm relying on API errors.
If a translation for WordPress core doesn't exist, the step will fail.

Theme and plugin translations are still allowed to fail because they have much lower translation rates.

adamziel · 2024-07-12T07:00:52Z

packages/playground/blueprints/src/lib/steps/set-site-language.ts

+
+	for (const { url, type } of translations) {
+		try {
+			const response = await fetch(url);


One last nit: Let's fetch using a promise queue instead of doing it sequentially

Actually, let's merge as is and do the promise queue in the PHP implementation.

adamziel · 2024-07-12T12:47:11Z

I added a usage example to the PR description and updated this Blueprint to use the new step:

https://github.com/WordPress/blueprints/blob/trunk/blueprints/translations/blueprint.json

Add setSiteLanguage step and language query param

64c4e12

bgrgicak self-assigned this Jun 24, 2024

bgrgicak requested review from adamziel and a team and removed request for adamziel June 24, 2024 07:51

bgrgicak added [Package][@wp-playground] Blueprints [Aspect] WordPress [Type] Enhancement New feature or request labels Jun 24, 2024

bgrgicak mentioned this pull request Jun 24, 2024

(Meetup use case) Display WP Playground interface in browser language #1513

Closed

bgrgicak changed the title ~~Add setSiteLanguage step and language query param~~ Jun 24, 2024

bgrgicak changed the title ~~Add langauge selection support~~ Jun 24, 2024

adamziel reviewed Jun 25, 2024

View reviewed changes

packages/playground/blueprints/src/lib/steps/set-site-language.ts Outdated Show resolved Hide resolved

bgrgicak force-pushed the add/set-site-language-step branch 2 times, most recently from a719571 to 8cc084f Compare July 5, 2024 10:25

bgrgicak requested a review from adamziel July 8, 2024 07:51

bgrgicak marked this pull request as draft July 9, 2024 05:48

bgrgicak added 3 commits July 11, 2024 08:02

Merge branch 'trunk' into add/set-site-language-step

b3235e8

Add comments

c09a2dc

Move set language to end of the list

01bd158

bgrgicak added 3 commits July 11, 2024 10:04

Fetch all translations from downloads.wordpress.org

8b1357e

Prevent step from failing if translations don't exist

f2325fa

Merge branch 'trunk' into add/set-site-language-step

b1182ed

bgrgicak marked this pull request as ready for review July 11, 2024 08:27

bgrgicak requested a review from akirk July 11, 2024 08:34

adamziel reviewed Jul 11, 2024

View reviewed changes

packages/playground/blueprints/public/blueprint-schema.json Outdated Show resolved Hide resolved

akirk reviewed Jul 11, 2024

View reviewed changes

bgrgicak added 2 commits July 12, 2024 05:55

Merge branch 'trunk' into add/set-site-language-step

07a27da

Throw error if core translations weren't found

d0987e7

bgrgicak commented Jul 12, 2024

View reviewed changes

packages/playground/blueprints/public/blueprint-schema.json Outdated Show resolved Hide resolved

bgrgicak commented Jul 12, 2024

View reviewed changes

Apply suggestions from code review

1459d40

bgrgicak requested a review from adamziel July 12, 2024 04:42

Remove test

600e22c

adamziel reviewed Jul 12, 2024

View reviewed changes

adamziel approved these changes Jul 12, 2024

View reviewed changes

adamziel changed the title ~~Add language selection support~~ Jul 12, 2024

adamziel merged commit c7fac48 into trunk Jul 12, 2024
5 checks passed

adamziel deleted the add/set-site-language-step branch July 12, 2024 11:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Blueprints: Add setSiteLanguage step to change the language #1538

Blueprints: Add setSiteLanguage step to change the language #1538

bgrgicak commented Jun 24, 2024 •

edited by adamziel

Loading

bgrgicak commented Jun 24, 2024

akirk commented Jun 25, 2024 •

edited

Loading

bgrgicak commented Jun 26, 2024

akirk commented Jun 26, 2024

brandonpayton commented Jun 27, 2024 •

edited

Loading

brandonpayton commented Jun 27, 2024

brandonpayton commented Jun 27, 2024 •

edited

Loading

bgrgicak commented Jun 28, 2024

bgrgicak commented Jun 28, 2024

bgrgicak commented Jul 3, 2024

bgrgicak commented Jul 8, 2024

pkevan commented Jul 8, 2024

bgrgicak commented Jul 8, 2024

pkevan commented Jul 8, 2024

bgrgicak commented Jul 11, 2024

akirk commented Jul 11, 2024

bgrgicak commented Jul 11, 2024

bgrgicak commented Jul 11, 2024

akirk commented Jul 11, 2024

bgrgicak commented Jul 11, 2024

adamziel commented Jul 11, 2024

akirk left a comment

bgrgicak Jul 12, 2024

adamziel Jul 12, 2024

adamziel Jul 12, 2024

adamziel commented Jul 12, 2024

Blueprints: Add setSiteLanguage step to change the language #1538

Blueprints: Add setSiteLanguage step to change the language #1538

Conversation

bgrgicak commented Jun 24, 2024 • edited by adamziel Loading

Motivation for the change, related issues

Implementation details

Testing Instructions (or ideally a Blueprint)

bgrgicak commented Jun 24, 2024

akirk commented Jun 25, 2024 • edited Loading

bgrgicak commented Jun 26, 2024

akirk commented Jun 26, 2024

brandonpayton commented Jun 27, 2024 • edited Loading

brandonpayton commented Jun 27, 2024

brandonpayton commented Jun 27, 2024 • edited Loading

bgrgicak commented Jun 28, 2024

bgrgicak commented Jun 28, 2024

bgrgicak commented Jul 3, 2024

bgrgicak commented Jul 8, 2024

pkevan commented Jul 8, 2024

bgrgicak commented Jul 8, 2024

pkevan commented Jul 8, 2024

bgrgicak commented Jul 11, 2024

akirk commented Jul 11, 2024

bgrgicak commented Jul 11, 2024

bgrgicak commented Jul 11, 2024

akirk commented Jul 11, 2024

bgrgicak commented Jul 11, 2024

adamziel commented Jul 11, 2024

akirk left a comment

Choose a reason for hiding this comment

bgrgicak Jul 12, 2024

Choose a reason for hiding this comment

adamziel Jul 12, 2024

Choose a reason for hiding this comment

adamziel Jul 12, 2024

Choose a reason for hiding this comment

adamziel commented Jul 12, 2024

bgrgicak commented Jun 24, 2024 •

edited by adamziel

Loading

akirk commented Jun 25, 2024 •

edited

Loading

brandonpayton commented Jun 27, 2024 •

edited

Loading

brandonpayton commented Jun 27, 2024 •

edited

Loading