Golang国际化(i18n)和本地化(l10n)指南

李海彬

发布于 2018-07-26 09:47:18

5.5K1

发布于 2018-07-26 09:47:18

文章被收录于专栏：Golang语言社区

原文作者：Theofanis Despoudis

Read the original article

Go is a statically compiled language that gained a lot of popularity lately due to the fact that is simple, performant and fits really well with developing cloud applications. It has a strong, yet poorly documented sub-package level base library that deals with a lot of aspects related to internationalization (i18n) and localization (l10n), such as character encodings, text transformations, and locale-specific text handling. Let's see what we can do to master this library and make our Go applications locale aware.

The package we are referring to is thegolang.org/x/text and if utilized correctly you can be pretty much cover a lot of parts when it comes to globalizing your apps. It comes with a set of abstractions to make easier to work with translatable messages, formatting, plural rules, Unicode and much more.

This article is going to consist of 2 parts. The first part is an overview of the golang.org/x/text package and the utilities it provides in terms of formatting and localization. Go excels at building microservice based architectures so in the second part, in order not to break this tradition, we are going to make a localization Server microservice that will help us understand the big picture of i18n and l10n support in Go.

For the purposes of this tutorial, I will be using the latest Go v1.10 and the code for this tutorial is hosted on Github.

Let’s get going.

Overview Of The Package

Most messages in Go programs pass through either the fmt or one of the template packages. The golang.org/x/text consists of multiple levels of sub-packages that offer lots of utilities and functions to format localized strings using a fmt style API. Let’s see how we can use it in practice.

Messages And Catalogs

A message is some form of content to be conveyed to the user. Each message is identified by a key, which can have many forms. You can create a message printer like that:

1p := message.NewPrinter(language.BritishEnglish)
2p.Printf("There are %v flowers in our garden.", 1500)

You need to supply a Language Tag when you call the NewPrinter function. Language tags are used whenever you want to specify a language. There are many ways you can create a tag such as:

Using predefined tags. For example:

1language.Greek, language.BrazilianPortuguese

The whole list of predefined tags is listed here.

From a string value. For example:

1language.Make("el"), language.Parse("en-UK")

By composing parts of type Tag, Base, Script, Region, Variant, []Variant, Extension, []Extension or error. For example:

1ja, _ := language.ParseBase("ja")
2jp, _ := language.ParseRegion("JP")
3jpLngTag, _ := language.Compose(ja, jp)
4fmt.Println(jpLngTag) // prints ja-JP

If you specify an invalid language tag you will get an instance of the Und Tag which denotes an Undefined Tag.

1fmt.Println(language.Compose(language.ParseRegion("AL"))) // prints Und-AL

f you want to learn more about the language API seethis doc here.

Coming back to our messages we can assign a new printer using a different language and print the formatted strings. The library will take care any localized formatting variants for you:

 1package main
 2import (
 3 "golang.org/x/text/message"
 4 "golang.org/x/text/language"
 5)
 6func main()  {
 7 p := message.NewPrinter(language.BritishEnglish)
 8 p.Printf("There are %v flowers in our garden.\n", 1500)
 9 p = message.NewPrinter(language.Greek)
10 p.Printf("There are %v flowers in our garden.", 1500)
11}

If you run this program you will get:

1$ go run main.go
2There are 1,500 flowers in our garden.
3There are 1.500 flowers in our garden.

Now in order to print translated messages, we need to add them to the message catalog so that the Printer can find them for the right language tag.

A Catalog defines collections of translated format strings. Think of it as a set of per-language dictionaries with translations for a set of keys. In order to use catalogs, we need to populate them with translations.

In practice, translations will be automatically injected from a translator-supplied data source. Let’s see how we can do it manually:

 1package main
 2import (
 3 "golang.org/x/text/message"
 4 "golang.org/x/text/language"
 5 "fmt"
 6)
 7func init()  {
 8 message.SetString(language.Greek, "%s went to %s.",  "%s πήγε στήν %s.")
 9 message.SetString(language.AmericanEnglish, "%s went to %s.",  "%s is in %s.")
10 message.SetString(language.Greek, "%s has been stolen.",  "%s κλάπηκε.")
11 message.SetString(language.AmericanEnglish, "%s has been stolen.",  "%s has been stolen.")
12        message.SetString(language.Greek, "How are you?", "Πώς είστε?.")
13}
14func main()  {
15 p := message.NewPrinter(language.Greek)
16 p.Printf("%s went to %s.", "Ο Πέτρος", "Αγγλία")
17 fmt.Println()
18 p.Printf("%s has been stolen.", "Η πέτρα")
19 fmt.Println()
20 p = message.NewPrinter(language.AmericanEnglish)
21 p.Printf("%s went to %s.", "Peter", "England")
22 fmt.Println()
23 p.Printf("%s has been stolen.", "The Gem")
24}

If you run this program you will get the following output:

1$ go run main.go
2Ο Πέτρος πήγε στήν Αγγλία.
3Η πέτρα κλάπηκε.
4Peter is in England.
5The Gem has been stolen.%

Caution: The keys you specify when you use the SetString method are case and line sensitive, which means that if you try to use PrintLn or add an end of line char \n then it won’t work:

1p := message.NewPrinter(language.Greek)
2p.Printf("%s went to %s.\n", "Ο Πέτρος", "Αγγλία") // will print Ο Πέτρος went to Αγγλία.
3p.Println("How are you?") // will print How are you?

Typically you don’t create catalogs but let the library handle them for you. You can also have the option to build ones programmatically using the catalog.Builder function.

Handling Plurals

For cases when you need to add multiple string translations depending on plural values, you need to add special calls to configure that in your translation catalogs. The sub-package golang.org/x/text/feature/plural exposes a function called SelectF that is used to define multiple linguistic plurals in a text.

I give below some typical usages of this function:

 1func init() {
 2        message.Set(language.Greek, "You have %d. problem",
 3 plural.Selectf(1, "%d",
 4 "=1", "Έχεις ένα πρόβλημα",
 5 "=2", "Έχεις %[1]d πρόβληματα",
 6 "other", "Έχεις πολλά πρόβληματα",
 7 ))
 8        message.Set(language.Greek, "You have %d days remaining",
 9 plural.Selectf(1, "%d",
10 "one", "Έχεις μία μέρα ελεύθερη",
11 "other", "Έχεις %[1]d μέρες ελεύθερες",
12 ))
13}
14func main()  {
15 p := message.NewPrinter(language.Greek)
16 p.Printf("You have %d. problem", 1)
17 fmt.Println()
18 p.Printf("You have %d. problem", 2)
19 fmt.Println()
20 p.Printf("You have %d. problem", 5)
21 fmt.Println()
22 p.Printf("You have %d days remaining", 1)
23 fmt.Println()
24 p.Printf("You have %d days remaining", 10)
25 fmt.Println()
26}

If you run this program you will get the following output:

1$ go run main.go
2Έχεις ένα πρόβλημα
3Έχεις 2 πρόβληματα
4Έχεις πολλά πρόβληματα
5Έχεις μία μέρα ελεύθερη
6Έχεις 10 μέρες ελεύθερες

The cases as provided in this function can support several variations such as zero , one , two , few , many and it can also match comparisons such as >x or <x.

String Interpolation In Messages

In some other cases where you want to handle further possible variants of a message, you can assign placeholder variables that can handle some specific cases of linguistic features. For instance, in the previous example where we used the plural can be written as:

 1func init() {
 2        message.Set(language.Greek, "You are %d minute(s) late.",
 3 catalog.Var("minutes", plural.Selectf(1, "%d", "one", "λεπτό", "other", "λεπτά")),
 4 catalog.String("Αργήσατε %[1]d ${minutes}."))
 5}
 6func main()  {
 7 p := message.NewPrinter(language.Greek)
 8 p.Printf("You are %d minute(s) late.", 1) // prints Αργήσατε 1 λεπτό
 9 fmt.Println()
10 p.Printf("You are %d minute(s) late.", 10)// prints Αργήσατε 10 λεπτά
11 fmt.Println()
12}

The catalog.Var assigns a special tag to the first string parameter minutes so it can be substituted with a more relevant translation based on the value of the %d parameter.

Formatting Currency

Package golang.org/x/text/currency deals with currency formatting rules. For currency, there are some useful functions to print locale-specific strings regarding amounts. For example here are some ways you can format them:

 1package main
 2import (
 3 "golang.org/x/text/message"
 4 "golang.org/x/text/language"
 5 "fmt"
 6 "golang.org/x/text/currency"
 7)
 8func main()  {
 9        p := message.NewPrinter(language.English)
10        p.Printf("%d", currency.Symbol(currency.USD.Amount(0.1)))
11 fmt.Println()
12 p.Printf("%d", currency.NarrowSymbol(currency.JPY.Amount(1.6)))
13 fmt.Println()
14 p.Printf("%d", currency.ISO.Kind(currency.Cash)(currency.EUR.Amount(12.255)))
15 fmt.Println()

And the result will be:

1$ go run main.go  
2US$ 0.10
3¥ 2
4EUR 12.26

Loading Messages

When you work with translations typically you will need to load the translations before so that the application can use them. You can think of those files as static resources. You have a few options on how you deploy those files with the application:

Manually Setting The Translation Strings

The simplest way to organize the translations is to have them assigned into the application binary. You will have to manually create an array of entries that will be used on init to load the messages into the default catalog. Then on your application, you only have to switch locale using the NewPrinter function.

Bellow is an example application by loading translations on init:

 1package main
 2import (
 3 "golang.org/x/text/language"
 4 "golang.org/x/text/feature/plural"
 5 "golang.org/x/text/message"
 6 "golang.org/x/text/message/catalog"
 7)
 8type entry struct {
 9 tag, key string
10 msg      interface{}
11}
12var entries = [...]entry{
13 {"en", "Hello World", "Hello World"},
14 {"el", "Hello World", "Για Σου Κόσμε"},
15 {"en", "%d task(s) remaining!", plural.Selectf(1, "%d",
16 "=1", "One task remaining!",
17 "=2", "Two tasks remaining!",
18 "other", "[1]d tasks remaining!",
19 )},
20 {"el", "%d task(s) remaining!", plural.Selectf(1, "%d",
21 "=1", "Μία εργασία έμεινε!",
22 "=2", "Μια-δυο εργασίες έμειναν!",
23 "other", "[1]d εργασίες έμειναν!",
24 )},
25}
26func init()  {
27 for _, e := range entries {
28     tag := language.MustParse(e.tag)
29     switch msg := e.msg.(type) {
30     case string:
31         message.SetString(tag, e.key, msg)
32     case catalog.Message:
33         message.Set(tag, e.key, msg)
34     case []catalog.Message:
35         message.Set(tag, e.key, msg...)
36     }
37 }
38}
39func main()  {
40 p := message.NewPrinter(language.Greek)
41 p.Printf("Hello World")
42 p.Println()
43 p.Printf("%d task(s) remaining!", 2)
44 p.Println()
45 p = message.NewPrinter(language.English)
46 p.Printf("Hello World")
47 p.Println()
48 p.Printf("%d task(s) remaining!", 2)
49}

If you run this program then it will print:

1$ go run examples/static/main.go         
2Για Σου Κόσμε
3Μια-δυο εργασίες έμειναν!
4Hello World
5Two tasks remaining!%

In practice, while this way is simple to implement, it’s not scalable enough. It works only for small applications with few translations. You will have to manually set the translation strings and it’s tricky to automate. For all other reasons, it’s recommended to automatically load messages where I explain in detail how to do it next.

Automatic Loading Of Messages

Traditionally, most localization frameworks have grouped data in per-language dynamically-loaded files. You can distribute those files to translators and have them merged into your app when they are ready.

To assist in this process the authors have included a helper CLI tool called gotext that is used for managing text in Go source code.

Let’s start by making sure that you have the latest version:

1$ go get -u golang.org/x/text/cmd/gotext

Running this tool will only show the options available and the help switch will not show any other info:

1$ gotext                        
2gotext is a tool for managing text in Go source code.
3Usage:
4        gotext command [arguments]
5The commands are:
6        update      merge translations and generate catalog
7        extract     extracts strings to be translated from code
8        rewrite     rewrites fmt functions to use a message Printer
9        generate    generates code to insert translated messages

For the purposes of this tutorial let’s use the update flag which performs a multi-step process of extracting the translation keys to a file and updating the code for loading them into catalogs for ease of use.

Create a file main.go and add a few PrintF calls and make sure you include the comment for the go:generate command

1$ touch main.go

File: main.go

 1package main
 2//go:generate gotext -srclang=en update -out=catalog/catalog.go -lang=en,el
 3import (
 4 "golang.org/x/text/language"
 5 "golang.org/x/text/message"
 6)
 7func main() {
 8 p := message.NewPrinter(language.Greek)
 9 p.Printf("Hello world!")
10 p.Println()
11 p.Printf("Hello", "world!")
12 p.Println()
13 person := "Alex"
14 place := "Utah"
15 p.Printf("Hello ", person, " in ", place, "!")
16 p.Println()
17 // Greet everyone.
18 p.Printf("Hello world!")
19 p.Println()
20 city := "Munich"
21 p.Printf("Hello %s!", city)
22 p.Println()
23 // Person visiting a place.
24 p.Printf("%s is visiting %s!",
25 person,
26 place)
27 p.Println()
28 // Double arguments.
29 miles := 1.2345
30 p.Printf("%.2[1]f miles traveled (%[1]f)", miles)
31}

Run the following commands:

1$ mkdir catalog
2$ go generate

Then fix the import to include the catalog.go file:

*File: main.go

1package main
2//go:generate gotext -srclang=en update -out=catalog/catalog.go -lang=en,el
3import (
4 "golang.org/x/text/language"
5 "golang.org/x/text/message"
6      _ "golang.org/x/text/message/catalog"
7)
8...

Now if you see the project structure there are some files created:

 1$ tree .
 2.
 3├── catalog
 4│   └── catalog.go
 5├── locales
 6│   ├── el
 7│   │   └── out.gotext.json
 8│   └── en
 9│       └── out.gotext.json
10├── main.go

The locales folder contain the translation messages in the format that the library supports. Typically you want to provide translations for this. Create a new file named messages.gotext.json and provide translations for the Greek language.

1$ touch locales/el/messages.gotext.json

File: locales/el/messages.gotext.json

 1{
 2  "language": "el",
 3  "messages": [
 4    {
 5      "id": "Hello world!",
 6      "message": "Hello world!",
 7      "translation": "Γιά σου Κόσμε!"
 8    },
 9    {
10      "id": "Hello",
11      "message": "Hello",
12      "translation": "Γιά σας %[1]v",
13      "placeholders": [
14        {
15          "id": "World",
16          "string": "%[1]v",
17          "type": "string",
18          "underlyingType": "string",
19          "argNum": 1,
20          "expr": "\"world!\""
21        }
22      ]
23    },
24    {
25      "id": "Hello {City}!",
26      "message": "Hello {City}!",
27      "translation": "Γιά σου %[1]s",
28      "placeholders": [
29        {
30          "id": "City",
31          "string": "%[1]s",
32          "type": "string",
33          "underlyingType": "string",
34          "argNum": 1,
35          "expr": "city"
36        }
37      ]
38    },
39    {
40      "id": "{Person} is visiting {Place}!",
41      "message": "{Person} is visiting {Place}!",
42      "translation": "Ο %[1]s επισκέπτεται την %[2]s",
43      "placeholders": [
44        {
45          "id": "Person",
46          "string": "%[1]s",
47          "type": "string",
48          "underlyingType": "string",
49          "argNum": 1,
50          "expr": "person"
51        },
52        {
53          "id": "Place",
54          "string": "%[2]s",
55          "type": "string",
56          "underlyingType": "string",
57          "argNum": 2,
58          "expr": "place"
59        }
60      ]
61    },
62    {
63      "id": "{Miles} miles traveled ({Miles_1})",
64      "message": "{Miles} miles traveled ({Miles_1})",
65      "translation": "%.2[1]f μίλια ταξίδεψε %[1]f",
66      "placeholders": [
67        {
68          "id": "Miles",
69          "string": "%.2[1]f",
70          "type": "float64",
71          "underlyingType": "float64",
72          "argNum": 1,
73          "expr": "miles"
74        },
75        {
76          "id": "Miles_1",
77          "string": "%[1]f",
78          "type": "float64",
79          "underlyingType": "float64",
80          "argNum": 1,
81          "expr": "miles"
82        }
83      ]
84    }
85  ]
86}

Now run the go generate command and the program next and see that the translations are happening:

1$ go generate
2$ go run main.go
3Γιά σου Κόσμε!
4Γιά σας world!
5Γιά σου Κόσμε!
6Γιά σου Munich
7Ο Alex επισκέπτεται την Utah
81,23 μίλια ταξίδεψε 1,234500%

In case you are interested, the rewrite flag searches for references to fmt in the source code and replaces them with the p.Print functions. For example, let’s say we have the following program:

File: main.go

1func main() {
2   p := message.NewPrinter(language.German)
3   fmt.Println("Hello world")
4   fmt.Printf("Hello world!")
5   p.Printf("Hello world!\n")
6}

If you run the following command:

1$ gotext rewrite -out main.go

Then the main.go will turn into:

1func main() {
2   p := message.NewPrinter(language.German)
3   p.Printf("Hello world\n")
4   p.Printf("Hello world!")
5   p.Printf("Hello world!\n")
6}

Example Microservice

This is the second part of the article where we can utilize in practice what we learned about the golang/x/text package. We are going to build a simple HTTP server that will serve an endpoint that will accept a user language parameter. Then It will try to match this parameter with the list of supported languages and then serve a translated response based on the most suitable locale.

First, make sure you have all dependencies installed.

Start by creating an application skeleton:

1$ go get -u github.com/golang/dep/cmd/dep
2$ dep init
3$ touch main.go

File: main.go

 1package main
 2import (
 3 "html"
 4 "log"
 5 "net/http"
 6        "fmt"
 7 "flag"
 8 "time"
 9)
10const (
11 httpPort  = "8090"
12)
13func PrintMessage(w http.ResponseWriter, r *http.Request) {
14 fmt.Fprintf(w, "Hello, %s", html.EscapeString(r.Host))
15}
16func main() {
17 var port string
18 flag.StringVar(&port, "port", httpPort, "http port")
19 flag.Parse()
20 server := &http.Server{
21 Addr:           ":" + port,
22 ReadTimeout:    10 * time.Second,
23 WriteTimeout:   10 * time.Second,
24 MaxHeaderBytes: 1 << 16,
25 Handler:        http.HandlerFunc(PrintMessage)}
26 log.Fatal(server.ListenAndServe())
27}

This example HTTP server does not handle translations yet. We can do that by replacing the call to fmt.FprintF with the call to p.FprintF

1func PrintMessage(w http.ResponseWriter, r *http.Request) {
2   p := message.NewPrinter(language.English)
3   p.Fprintf(w,"Hello, %v", html.EscapeString(r.Host))

Add the following line to your source code and run the go generate command:

1//go:generate gotext -srclang=en update -out=catalog/catalog.go -lang=en,el

1$ dep ensure -update
2$ go generate        
3el: Missing entry for "Hello, {Host}".

Provide translations for the missing entries:

1$ cp locales/el/out.gotext.json locales/el/messages.gotext.json

File: locales/el/messages.gotext.json

 1{
 2    "language": "el",
 3    "messages": [
 4        {
 5            "id": "Hello, {Host}",
 6            "message": "Hello, {Host}",
 7            "translation": "Γιά σου %[1]v",
 8            "placeholders": [
 9                {
10                    "id": "Host",
11                    "string": "%[1]v",
12                    "type": "string",
13                    "underlyingType": "string",
14                    "argNum": 1,
15                    "expr": "html.EscapeString(r.Host)"
16                }
17            ]
18        }
19    ]
20}

Run the command go generate again and add a reference to the catalog package in main.go :

1$ go generate

File: main.go

 1package main
 2import (
 3 "html"
 4 "log"
 5 "net/http"
 6 "flag"
 7 "time"
 8 "golang.org/x/text/message"
 9 "golang.org/x/text/language"
10 _ "go-internationalization/catalog"
11)
12...

Now in order to determine which language we need to switch when the user requests a resource from the API we need to add a Matcher object that will be used to determine the best match out of our supported locales when provided with a list of language tags. Create a new Matcher by providing the list of supported locales from the message.DefaultCatalog that is populated from the gotext tool:

File: main.go

1var matcher = language.NewMatcher(message.DefaultCatalog.Languages())

Add your function to match the correct language based on the request parameters:

File: main.go

 1package main
 2import (
 3 "html"
 4 "log"
 5 "net/http"
 6 "flag"
 7 "time"
 8        "context"
 9 "golang.org/x/text/message"
10 "golang.org/x/text/language"
11 _ "go-internationalization/catalog"
12)
13//go:generate gotext -srclang=en update -out=catalog/catalog.go -lang=en,el
14var matcher = language.NewMatcher(message.DefaultCatalog.Languages())
15type contextKey int
16const (
17 httpPort  = "8090"
18 messagePrinterKey contextKey = 1
19)
20func withMessagePrinter(next http.HandlerFunc) http.HandlerFunc {
21 return func(w http.ResponseWriter, r *http.Request) {
22     lang, ok := r.URL.Query()["lang"]
23     if !ok || len(lang) < 1 {
24         lang = append(lang, language.English.String())
25     }
26     tag,_, _ := matcher.Match(language.MustParse(lang[0]))
27     p := message.NewPrinter(tag)
28     ctx := context.WithValue(context.Background(), messagePrinterKey, p)
29     next.ServeHTTP(w, r.WithContext(ctx))
30 }
31}
32...

I only supplied a parameter parsed from the query string. You can mix and match also additional tags parsed from a cookie or an Accept-Language header.

Now you only need to wrap your handler function PrintMessage with the withMessagePrinter and grab the printer from the context:

File: main.go

 1...
 2func PrintMessage(w http.ResponseWriter, r *http.Request) {
 3 p := r.Context().Value(messagePrinterKey).(*message.Printer)
 4 p.Fprintf(w,"Hello, %v", html.EscapeString(r.Host))
 5}
 6func main() {
 7 var port string
 8 flag.StringVar(&port, "port", httpPort, "http port")
 9 flag.Parse()
10 server := &http.Server{
11 Addr:           ":" + port,
12 ReadTimeout:    10 * time.Second,
13 WriteTimeout:   10 * time.Second,
14 MaxHeaderBytes: 1 << 16,
15 Handler:        http.HandlerFunc(withMessagePrinter(PrintMessage))}
16 log.Fatal(server.ListenAndServe())
17}

Start the server and issue some requests to see the translations happening:

1$ go run main.go

The world is your oyster from now on…

Use Phraseapp

PhraseApp supports many different languages and frameworks, including Go. It allows to easily import and export translations data and search for any missing translations, which is really convenient. On top of that, you can collaborate with translators as it is much better to have professionally done localization for your website. If you’d like to learn more about PhraseApp, refer to the Getting Started guide. You can also get a 14-days trial. So what are you waiting for?

Conclusion

In this article, we explored how Go manages localization using the golang/x/text package and we implemented an example web server that serves translations and explaining how all pieces fit together. As the official documentation lacks in terms of practical examples, I hope that this article explained out the principles of adding i18n, in a simple manner, to your Go applications and could be used as a good future reference.

This is by no means an exhaustive guide as every application has different needs and scope requirements. Please stay put for more detailed articles regarding this subject.

版权申明：内容来源网络，版权归原创者所有。除非无法确认，我们都会标明作者及出处，如有侵权烦请告知，我们会立即删除并表示歉意。谢谢。

本文参与腾讯云自媒体同步曝光计划，分享自微信公众号。

原始发表：2018-06-04，如有侵权请联系 cloudcommunity@tencent.com 删除

其他

本文分享自 Golang语言社区微信公众号，前往查看

如有侵权，请联系 cloudcommunity@tencent.com 删除。

本文参与腾讯云自媒体同步曝光计划，欢迎热爱写作的你一起参与！

其他