Skip to content

Commit

Permalink
Implement use_keys_as_context feature (see #89)
Browse files Browse the repository at this point in the history
  • Loading branch information
Igor Afanasyev committed Jul 22, 2019
1 parent 6ef1799 commit 172c214
Show file tree
Hide file tree
Showing 38 changed files with 583 additions and 0 deletions.
11 changes: 11 additions & 0 deletions doc/sample.serge
View file
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,17 @@ jobs
# NO - the string is extracted as is (default)
normalize_strings NO

# (BOOLEAN) [OPTIONAL] Should the string keys reported by a parser
# be copied into a context field?
# Note that original context in this case becomes overwritten,
# unless the returned key is empty.
# Changing this setting on an existing previously translated project
# will change all the string signatures and may require one to re-apply
# and review translations once.
# YES - extracted keys should be used as context
# NO - keep the original extracted context (default)
use_keys_as_context NO

# (BOOLEAN) [OPTIONAL] Should untranslated strings appear blank
# in the output file, or have source strings as their "translation"?
# Rendering source string when a translation is missing may simplify
Expand Down
8 changes: 8 additions & 0 deletions lib/Serge/Engine.pm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -773,6 +773,14 @@ sub disambiguate_string {
$self->{current_file_source_keys}->{$source_key} = 1;
}

# if job's `use_keys_as_context` option is turned on,
# and the context has not been provided explicitly,
# copy the key into the context before disambiguating

if ($context eq '' && $self->{job}->{use_keys_as_context} && $source_key ne '') {
$context = $source_key;
}

# see if the item was already found in this file and
# alter context if necessary to disambiguate the string

Expand Down
1 change: 1 addition & 0 deletions lib/Serge/config_schema.serge
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ jobs ARRAY
}

normalize_strings BOOLEAN
use_keys_as_context BOOLEAN
leave_untranslated_blank BOOLEAN
db_source STRING
db_username STRING
Expand Down
3 changes: 3 additions & 0 deletions t/data/engine/feature_branch/01-dont-use-keys-as-context/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Test for `feature_branch` plugin with `use_keys_as_context` turned OFF.
The test shows how the plugin behaves when translating identical strings
in the file (see https://github.com/evernote/serge/issues/89)
5 changes: 5 additions & 0 deletions t/data/engine/feature_branch/01-dont-use-keys-as-context/reference-output/database/files
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
files
{
0 1 1 test_job.master test_namespace strings.ini 0
1 2 8 test_job.feature1 test_namespace feature1/strings.ini 0
}
7 changes: 7 additions & 0 deletions t/data/engine/feature_branch/01-dont-use-keys-as-context/reference-output/database/items
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
items
{
0 1 3 1 1 key1 NO 0
1 2 5 1 2 key2 NO 0
2 3 9 2 1 key0 NO 0
3 4 11 2 3 key1 NO 0
}
44 changes: 44 additions & 0 deletions ...ta/engine/feature_branch/01-dont-use-keys-as-context/reference-output/database/properties
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
properties
{
0 1 source:1 5b9aca4d3c50d6a78e64a1a174706159
1 2 hash:1 5b9aca4d3c50d6a78e64a1a174706159
2 3 size:1 18
3 4 items:1 1,2
4 5 ts:1:test:count 2
5 6 usn:1:test 7
6 7 ts:1:test 78ef34e963474936a92ed5894f3f77ed
7 8 target:1:test_job.master:test
745ab09bc775f5d4698d0ebb5e8e9f2a
8 9 target:mtime:1:test_job.master:test 12345678
9 10 source:1:test_job.master:test
5b9aca4d3c50d6a78e64a1a174706159
10 11 source:ts:1:test_job.master:test
78ef34e963474936a92ed5894f3f77ed
11 12 job-hash:test_namespace:test_job.master
101680447a2f0667f875179c842bc296
12 13 job-plugin:test_namespace:test_job.master parse_keyvalue.
13 14 job-serializer-plugin:test_namespace:test_job.master
serialize_po.
14 15 job-engine:test_namespace:test_job.master 1.3.2
15 16 source:2 b1980d771cb73f4e7c5b339d53f7f00b
16 17 hash:2 b1980d771cb73f4e7c5b339d53f7f00b
17 18 size:2 27
18 19 items:2 3,4
19 20 ts:2:test:count 2
20 21 usn:2:test 13
21 22 ts:2:test 6653cc0e10fa61bfcd4d1683e8b4a672
22 23 target:2:test_job.feature1:test
ada57d5b7982a0806ab74c1fc8ca9d89
23 24 target:mtime:2:test_job.feature1:test 12345678
24 25 source:2:test_job.feature1:test
b1980d771cb73f4e7c5b339d53f7f00b
25 26 source:ts:2:test_job.feature1:test
6653cc0e10fa61bfcd4d1683e8b4a672
26 27 job-hash:test_namespace:test_job.feature1
2e7d9c998cff60c372655f124b0390a2
27 28 job-plugin:test_namespace:test_job.feature1
parse_keyvalue.
28 29 job-serializer-plugin:test_namespace:test_job.feature1
serialize_po.
29 30 job-engine:test_namespace:test_job.feature1 1.3.2
}
6 changes: 6 additions & 0 deletions t/data/engine/feature_branch/01-dont-use-keys-as-context/reference-output/database/strings
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
strings
{
0 1 2 foo NO 0
1 2 4 foo key2 0
2 3 10 foo key1 0
}
7 changes: 7 additions & 0 deletions .../engine/feature_branch/01-dont-use-keys-as-context/reference-output/database/translations
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
translations
{
0 1 6 1 test ḟőő NO 0 0
1 2 7 2 test ḟőő NO 0 0
2 3 12 3 test ḟőő NO 0 0
3 4 13 4 test ḟőő NO 0 0
}
3 changes: 3 additions & 0 deletions ...1-dont-use-keys-as-context/reference-output/localized-resources/feature1/test/strings.ini
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
key0=ḟőő
key1=ḟőő
key2=ḟőő
2 changes: 2 additions & 0 deletions ..._branch/01-dont-use-keys-as-context/reference-output/localized-resources/test/strings.ini
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
key1=ḟőő
key2=ḟőő
19 changes: 19 additions & 0 deletions ...ature_branch/01-dont-use-keys-as-context/reference-output/po/test/feature1/strings.ini.po
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Language: test\n"
"Generated-By: Serge 1.3\n"

#. key0
#: File: feature1/strings.ini
#: ID: b9654bb58db28f9c09099c29f2897d78
msgid "foo"
msgstr "ḟőő"

#. key1
#: File: feature1/strings.ini
#: ID: 55cf3629c1cbfb182802ff6c106b88ee
msgctxt "key1"
msgid "foo"
msgstr "ḟőő"
19 changes: 19 additions & 0 deletions ...engine/feature_branch/01-dont-use-keys-as-context/reference-output/po/test/strings.ini.po
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Language: test\n"
"Generated-By: Serge 1.3\n"

#. key1
#: File: strings.ini
#: ID: b9654bb58db28f9c09099c29f2897d78
msgid "foo"
msgstr "ḟőő"

#. key2
#: File: strings.ini
#: ID: 458d67e28e3c72583d476a32fd00581d
msgctxt "key2"
msgid "foo"
msgstr "ḟőő"
3 changes: 3 additions & 0 deletions t/data/engine/feature_branch/01-dont-use-keys-as-context/resources/feature1/strings.ini
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
key0=foo
key1=foo
key2=foo
2 changes: 2 additions & 0 deletions t/data/engine/feature_branch/01-dont-use-keys-as-context/resources/master/strings.ini
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
key1=foo
key2=foo
62 changes: 62 additions & 0 deletions t/data/engine/feature_branch/01-dont-use-keys-as-context/translate.serge
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
inc
{
job-common
{
@inherit ../../common.serge#job_template
source_match \.ini$

parser
{
plugin parse_keyvalue
}

callback_plugins
{
-:test_language

:feature_branch
{
plugin feature_branch

data
{
master_job test_job.master
}
}

:test_language_lower_priority
{
plugin test_language
phase get_translation_pre

data
{
save_translations YES
}
}
}
}
}

jobs
{
# 'master branch' resources
:master
{
@inherit .#inc/job-common
source_dir ./resources/master
id test_job.master
}

# 'feature branch #1' resources
# (will only extract strings which are not in the 'test_job.master' job)
:feature1
{
@inherit .#inc/job-common
source_dir ./resources/feature1
source_path_prefix feature1/
output_file_path ./test-output/localized-resources/feature1/%LOCALE%/%FILE%
id test_job.feature1
}

}
3 changes: 3 additions & 0 deletions t/data/engine/feature_branch/02-use-keys-as-context/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Test for `feature_branch` plugin with `use_keys_as_context` turned ON.
The test shows how the plugin behaves when translating identical strings
in the file (see https://github.com/evernote/serge/issues/89)
5 changes: 5 additions & 0 deletions t/data/engine/feature_branch/02-use-keys-as-context/reference-output/database/files
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
files
{
0 1 1 test_job.master test_namespace strings.ini 0
1 2 8 test_job.feature1 test_namespace feature1/strings.ini 0
}
6 changes: 6 additions & 0 deletions t/data/engine/feature_branch/02-use-keys-as-context/reference-output/database/items
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
items
{
0 1 3 1 1 key1 NO 0
1 2 5 1 2 key2 NO 0
2 3 10 2 3 key0 NO 0
}
44 changes: 44 additions & 0 deletions t/data/engine/feature_branch/02-use-keys-as-context/reference-output/database/properties
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
properties
{
0 1 source:1 5b9aca4d3c50d6a78e64a1a174706159
1 2 hash:1 5b9aca4d3c50d6a78e64a1a174706159
2 3 size:1 18
3 4 items:1 1,2
4 5 ts:1:test:count 2
5 6 usn:1:test 7
6 7 ts:1:test e33b98515b2bf482638bed63f679fa40
7 8 target:1:test_job.master:test
745ab09bc775f5d4698d0ebb5e8e9f2a
8 9 target:mtime:1:test_job.master:test 12345678
9 10 source:1:test_job.master:test
5b9aca4d3c50d6a78e64a1a174706159
10 11 source:ts:1:test_job.master:test
e33b98515b2bf482638bed63f679fa40
11 12 job-hash:test_namespace:test_job.master
9c5438db01c139bcae8139b89e900f18
12 13 job-plugin:test_namespace:test_job.master parse_keyvalue.
13 14 job-serializer-plugin:test_namespace:test_job.master
serialize_po.
14 15 job-engine:test_namespace:test_job.master 1.3.2
15 16 source:2 b1980d771cb73f4e7c5b339d53f7f00b
16 17 hash:2 b1980d771cb73f4e7c5b339d53f7f00b
17 18 size:2 27
18 19 items:2 3
19 20 ts:2:test:count 1
20 21 usn:2:test 11
21 22 ts:2:test c35ff05da271adceae4240dc1353dccb
22 23 target:2:test_job.feature1:test
ada57d5b7982a0806ab74c1fc8ca9d89
23 24 target:mtime:2:test_job.feature1:test 12345678
24 25 source:2:test_job.feature1:test
b1980d771cb73f4e7c5b339d53f7f00b
25 26 source:ts:2:test_job.feature1:test
c35ff05da271adceae4240dc1353dccb
26 27 job-hash:test_namespace:test_job.feature1
67d4dfc0854662baaa2f15ef389d52c8
27 28 job-plugin:test_namespace:test_job.feature1
parse_keyvalue.
28 29 job-serializer-plugin:test_namespace:test_job.feature1
serialize_po.
29 30 job-engine:test_namespace:test_job.feature1 1.3.2
}
6 changes: 6 additions & 0 deletions t/data/engine/feature_branch/02-use-keys-as-context/reference-output/database/strings
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
strings
{
0 1 2 foo key1 0
1 2 4 foo key2 0
2 3 9 foo key0 0
}
6 changes: 6 additions & 0 deletions t/data/engine/feature_branch/02-use-keys-as-context/reference-output/database/translations
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
translations
{
0 1 6 1 test ḟőő NO 0 0
1 2 7 2 test ḟőő NO 0 0
2 3 11 3 test ḟőő NO 0 0
}
3 changes: 3 additions & 0 deletions ...nch/02-use-keys-as-context/reference-output/localized-resources/feature1/test/strings.ini
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
key0=ḟőő
key1=ḟőő
key2=ḟőő
2 changes: 2 additions & 0 deletions ...ature_branch/02-use-keys-as-context/reference-output/localized-resources/test/strings.ini
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
key1=ḟőő
key2=ḟőő
13 changes: 13 additions & 0 deletions ...ne/feature_branch/02-use-keys-as-context/reference-output/po/test/feature1/strings.ini.po
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Language: test\n"
"Generated-By: Serge 1.3\n"

#. key0
#: File: feature1/strings.ini
#: ID: 69e4142dd1335cad57a4934d6c88af63
msgctxt "key0"
msgid "foo"
msgstr "ḟőő"
20 changes: 20 additions & 0 deletions t/data/engine/feature_branch/02-use-keys-as-context/reference-output/po/test/strings.ini.po
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Language: test\n"
"Generated-By: Serge 1.3\n"

#. key1
#: File: strings.ini
#: ID: 55cf3629c1cbfb182802ff6c106b88ee
msgctxt "key1"
msgid "foo"
msgstr "ḟőő"

#. key2
#: File: strings.ini
#: ID: 458d67e28e3c72583d476a32fd00581d
msgctxt "key2"
msgid "foo"
msgstr "ḟőő"
3 changes: 3 additions & 0 deletions t/data/engine/feature_branch/02-use-keys-as-context/resources/feature1/strings.ini
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
key0=foo
key1=foo
key2=foo
2 changes: 2 additions & 0 deletions t/data/engine/feature_branch/02-use-keys-as-context/resources/master/strings.ini
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
key1=foo
key2=foo
Loading

0 comments on commit 172c214

Please sign in to comment.