python 中英文分离_利用Python将文本中的中英文分离方法

python 中英文分离_利用Python将文本中的中英文分离方法

2024-11-01 00:04

在进行文本分析、提取关键词时，新闻评论等文本通常是中英文及其他语言的混杂，若不加处理直接分析，结果往往差强人意。

下面对中英文文本进行分离做一下总结：

1、超短文本，ASCII识别。

result = "".join(i for i in s if ord(i) < 256)

print(result)

China's Legend Holdings will split its several business arms to go public on stock markets, the group's president Zhu Linan said on Tuesday.

2、unicode编码识别

import re

uncn = re.compile(r'[a-z, ]')

en = "".join(uncn.findall(s.lower()))

print(en)

chinas legend holdings will split its several business arms to go public on stock markets, the groups president zhu linan said on tuesday

中文的编码范围是：一-龥，相应的[^一-龥]可匹配非中文。