問題描述

ICU 項目(現在也有一個 PHP 庫) 包含幫助規范化 UTF-8 字符串所需的類，以便在搜索時更容易比較值.

The ICU project (which also now has a PHP library) contains the classes needed to help normalize UTF-8 strings to make it easier to compare values when searching.

但是，我正在嘗試弄清楚這對應用程序意味著什么.例如，在哪些情況下我需要規范等效"而不是兼容性等效"，或者反之亦然?

However, I'm trying to figure out what this means for applications. For example, in which cases do I want "Canonical Equivalence" instead of "Compatibility equivalence", or vis-versa?

推薦答案

關于 Unicode 規范化你從未想知道的一切

規范歸一化

Unicode 包括多種對某些字符進行編碼的方法，尤其是重音字符.規范歸一化將代碼點更改為規范編碼形式.除字體或渲染引擎中的任何錯誤外，生成的代碼點應與原始代碼點相同.

Everything You Never Wanted to Know about Unicode Normalization

Canonical Normalization

Unicode includes multiple ways to encode some characters, most notably accented characters. Canonical normalization changes the code points into a canonical encoding form. The resulting code points should appear identical to the original ones barring any bugs in the fonts or rendering engine.

因為結果看起來相同，所以在存儲或顯示字符串之前對字符串應用規范化始終是安全的，只要您能容忍結果與輸入不完全相同.

Because the results appear identical, it is always safe to apply canonical normalization to a string before storing or displaying it, as long as you can tolerate the result not being bit for bit identical to the input.

規范標準化有兩種形式:NFD 和 NFC.從可以在這兩種形式之間轉換而不會丟失的意義上說，這兩者是等效的.在 NFC 下比較兩個字符串將始終給出與在 NFD 下比較它們相同的結果.

Canonical normalization comes in 2 forms: NFD and NFC. The two are equivalent in the sense that one can convert between these two forms without loss. Comparing two strings under NFC will always give the same result as comparing them under NFD.

NFD 將字符完全展開.這是計算速度更快的歸一化形式，但會產生更多代碼點(即使用更多空間).

NFD has the characters fully expanded out. This is the faster normalization form to calculate, but the results in more code points (i.e. uses more space).

如果您只想比較兩個尚未標準化的字符串，這是首選的標準化形式，除非您知道需要兼容性標準化.

If you just want to compare two strings that are not already normalized, this is the preferred normalization form unless you know you need compatibility normalization.

NFC 會在運行 NFD 算法后盡可能重新組合代碼點.這需要更長的時間，但會產生更短的字符串.

NFC recombines code points when possible after running the NFD algorithm. This takes a little longer, but results in shorter strings.

Unicode 還包括許多真正不屬于的字符，但在遺留字符集中使用.Unicode 添加這些是為了允許將這些字符集中的文本作為 Unicode 處理，然后無損地轉換回來.

Unicode also includes many characters that really do not belong, but were used in legacy character sets. Unicode added these to allow text in those character sets to be processed as Unicode, and then be converted back without loss.

兼容性規范化將這些轉換為相應的真實"序列.字符，并且還執行規范歸一化.兼容性規范化的結果可能與原始結果不一致.

Compatibility normalization converts these to the corresponding sequence of "real" characters, and also performs canonical normalization. The results of compatibility normalization may not appear identical to the originals.

包含格式信息的字符將替換為不包含的字符.例如，字符 ? 被轉換為 9.其他不涉及格式差異.例如羅馬數字字符Ⅸ被轉換為常規字母IX.

Characters that include formatting information are replaced with ones that do not. For example the character ? gets converted to 9. Others don't involve formatting differences. For example the roman numeral character Ⅸ is converted to the regular letters IX.

顯然，一旦執行了這種轉換，就不可能再無損地轉換回原始字符集.

Obviously, once this transformation has been performed, it is no longer possible to losslessly convert back to the original character set.

Unicode Consortium 建議將兼容性規范化考慮為 ToUpperCase 轉換.它在某些情況下可能有用，但您不應隨意應用.

The Unicode Consortium suggests thinking of compatibility normalization like a ToUpperCase transform. It is something that may be useful in some circumstances, but you should not just apply it willy-nilly.

一個很好的用例是搜索引擎，因為您可能希望搜索 9 以匹配 ?.

An excellent use case would be a search engine since you would probably want a search for 9 to match ?.

您可能不應該做的一件事是向用戶顯示應用兼容性規范化的結果.

One thing you should probably not do is display the result of applying compatibility normalization to the user.

兼容性規范化形式有NFKD和NFKC兩種形式.它們與 NFD 和 C 之間的關系相同.

Compatibility normalization form comes in two forms NFKD and NFKC. They have the same relationship as between NFD and C.

NFKC 中的任何字符串本質上也在 NFC 中，對于 NFKD 和 NFD 也是如此.因此NFKD(x)=NFD(NFKC(x))，和NFKC(x)=NFC(NFKD(x))，等等

Any string in NFKC is inherently also in NFC, and the same for the NFKD and NFD. Thus NFKD(x)=NFD(NFKC(x)), and NFKC(x)=NFC(NFKD(x)), etc.

如果有疑問，請使用規范標準化.根據適用的空間/速度權衡選擇 NFC 或 NFD，或根據您要與之互操作的事物的要求選擇 NFC 或 NFD.

If in doubt, go with canonical normalization. Choose NFC or NFD based on the space/speed trade-off applicable, or based on what is required by something you are inter-operating with.

這篇關于什么是標準化的 UTF-8?的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！