My struggle with translations

“It’s day 15 of looking at translatability of desktop files that don’t have .desktop extension, I’ve lost all hope…”
-me after spending a lot of time on translations

Work till now

In my last post, after refactoring the code, I succeeded in making a Firmware and RetroFirmware in the overall scheme of making a Firmware object that has the information it needs and handles it independently. It also allows Firmware object to handle it’s checksum verification independently. Some inconsistencies regarding checksums in the core descriptor files were also addressed and solved.

Update… maybe

After I made Firmware and RetroFirmware in my previous post, what I needed to do was quite simple initially. Add a new “name” field to each [Firmware] group entry in every .libretro core descriptor file that has a [Firmware] group and write a parser for it in retro-gtk. But only writing the parser is not enough (like I did when I first pushed the code :p). The whole point of adding a new field is to have a translatable name, which means the parser needs to get the name string in the user’s region’s language. Which can then later be used by gnome-games in the UI.

Writing the parser was not that hard since there were examples in the retro-gtk. The only thing different from the other parser would be that this new one would get string from the key file depending on which locale the user is.

basically use

g_key_file_get_locale_string ()

instead of

g_key_file_get_string ()

with the parser complete, I just had to make some changes to the flatpak-manifest on my local gnome-games so that it uses my branch of retro-gtk, and see whether the parser functions or not.

Viola, the parser works and the strings in the new “name” field show up in the UI, easy. Or so I though. Then I tried testing after changing my system language. And it didn’t work. I tried a dozen of times after changing the parser, the tag and function calls. Then after discussing with my mentor, it turns out that translations don’t really work well when using builder :/

The only way to test now was to install the code to user. And after the 20 min it took to install on user, the translations finally worked. The parser was working as intended by changing the firmware names in the UI when the system language was changed. But later after discussing with my mentor, I realized that the translations team would not directly write these translations in the descriptors, they must use some application for convenience which will not detect the descriptor files.

Even when the parser is complete and functional, it does not mean that it’s usable. What needs to be done, is to link these descriptor files to gettext. When correctly linked with gettext, the translation entries would automatically show up in po files, and when translations are added to the po files, they will be added to the descriptors while building.

To link files to gettext, one only needs to add the file path to POTFILES.in, then gettext would automatically detect the type of file it is (source code or xml etc) and add the translatable entries to po files. But the thing with key files is that they are neither source code nor xml files. And especially because they are not xml files, a simple ITS rule is not enough when linking them to gettext because ITS rules are not applicable on custom or any desktop files.

As for gettext, when linking files with unknown extensions (such as the .libretro descriptor files), the file will be assumed and processsed as C source file. Which means that the keywords that need to be translated are missed (precisely because it’s not C source file). Another reason why the present gettext implementation is unusable is because if the current gettext implementation is forced to read these descriptor files as desktop files, it would completely break the existing translations.

The gettext infrastructure in gnome-games is not suitable for translation of core descriptor files. So, after some discussion with my mentor, I started looking for other gnome projects that might have already faced a similar issue and also contact someone from the translations team, to seek help regarding the matter.

And after some back and forth of emails in the mailing list, someone (not sure if I should name them here, but thank you for the help) sent a link to a public gist with a python script that handled translations of desktop files and some other commands that might help. With that script as example I wrote a script that does exactly what we wanted it to do with some side-effects. Link to my version of the script is here. What this script did when run in meson before anything was built was, update all present po files, take entries from po files and add them to the descriptor file.

As for the side-effects. The script would run even when running simple tests and not installing. Making changes to ~10 files every time the project was built regardless of whether the project is being tested and not installed on the system. A simple fix for this would be addition of the respective descriptor files to .gitignore. But another drawback to this script was that since the script did not know the existence of other files (source code or ui files), it kept messing with the existing translation. So to not mess with current translations, I made a new po directory just for core descriptor file translations. After sharing the experience with my mentor, I was advised not to make a new po directory as we didn’t know how the translation team handles translations for any given project and whether the new po directory would even be visible to them.

With the script path down the drain, I departed on a journey to find a project that does something similar. And as far as I could look, there was no similar project. Like a sailor stranded in open waters without any wind, I lost all hope. And that’s when my mentor found something that might be helpful. Repository of GTK on gitlab had 2 po directories. One for normal translation and another for translations of properties. This meant that two po directories in the same project is possible.

After making sure that the script works, I wanted to confirm that the gettext path was completely futile. Reason being the disadvantages of the script state above. If gettext were to work as intended, all issues will be resolved. So, while looking at how translations and gettext works I started working on linking the descriptors to a separate po directory.

Let’s start with meson’s I18n module. This module provides internationalization and localization functionalities. and basically has only two functions. everything related to importing and usage is on meson’s documentation [here]. As gnome-games uses meson, I needed to use this module to make a separate po directory, and link the translations to their the respective files.

First make a new directory with a meson.build file. with the following code.

i18n.gettext (meson.project_name (), preset: 'glib', args: ['-LDesktop'])

Add files to the POTFILES.in in this new dir, add a LINGUAS file. And BOOM, gettext recognises all the necessary fields and update/creates po files!

ALl that was left to do was merge translation entries from the po files with the core descriptor files using the following code.

libretro_cores = [
  'bsnes_mercury_balanced.libretro',
  'handy.libretro',
  'mednafen_pce_fast.libretro',
  'mednafen_saturn.libretro',
  'mgba.libretro',
  'nestopia.libretro',
  'parallel_n64.libretro',
  'pcsx_rearmed.libretro',
  'picodrive.libretro',
  'prosystem.libretro'
]

foreach libretro_core : libretro_cores
  i18n.merge_file (
    libretro_core,
    type: 'desktop',
    input: libretro_core,
    output: libretro_core,
    po_dir: libretro_podir
  )
endforeach

I’m sure that there would be a better way to do this. But it was all in vain. Because core descriptor files are directly used by their respective emulator before the project even builds. So, even if gettext were to work, it would be after the files are already used and hence would not serve it’s purpose. (ノಠ益ಠ)ノ彡┻━┻

After exhausting every choice I had, I shared my findings with my mentor and we decided to freeze the UI part of my project till 3.40 release. He also came up with an idea to make core descriptors a sub-project. This way, we have only one po directory in games (the po for core descriptors would be in a the sub-project). The sub-project would be installed using flatpak-manifest before their respective emulators are built. This would always make sure that the translations are there before games is built.

This was my struggle with translations with 2 weeks down the drain. Any how, these 2 weeks were very educational weeks.

To be continues in the next update…

One thought on “My struggle with translations

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create your website at WordPress.com
Get started
%d bloggers like this: