一分钟版本

1
2

>>> python xxx.py  # 直接运行
>>> python -m xxx # 相当于import，叫做当做模块来启动

主要区别在于sys.path不同

直接运行会将该脚本所在目录添加至sys.path
当做模块启动则会将当前运行命令的路径添加至sys.path

环境

• MacOS Sierra
• Python 3.6

安装参照使用pipenv管理你的项目

解决方案

在~/.bash_profile里加上下面两行

1
2

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

参考ValueError: unknown locale: UTF-8

原文地址

该文主要讲了GIL，在此记录自己的总结

最近项目中遇到一个问题，就是数据库中某些较早的记录中的中文名字显示为乱码，就像这样。

1

陳彥璋 # 陳彥璋

于是就寻思着怎样才能用Python3自动进行乱码恢复。

Why @property?

如果对于一个类，我们希望用getter和setter来控制成员变量的赋值和取值，又不希望对于每一个成员变量都显式的写出getter和setter， 就可以用@property

1
2
3
4
5
6
7
8
9
10
11
12

class Student:
    
    # this is the getter
    @property
    def score(self):
        return self._score
    
    # this is the setter
    @score.setter
    def score(self, value):
        # value check here
        self._score = value

When calling

1
2

Student().score # get_score
# instead of Student()._score directly

More details about the func property here

Reference

• How does the @property decorator work?
• 使用@property

如果我们总想要做到完美，一步到位，其实是在逃避成长啊。

txtaug

txtaug is a command line tool written in Python3 for text augmentation based on Baidu translation API.

需求

• 希望得到大量（几乎全部）符合一定搜索条件的推文

现状

• 推特自身的API限制诸多，免费API每次请求只能得到100条，只能得到最近7天的推文。付费API虽然能检索full archive，但时间窗口被限定为30天，即只能检索过去所有推文中任意一个30天窗口内的推文…

需求

最近要处理香港地铁相关的推文。推文中除了英文外还包含中文，日文，泰文等语言，同时一个句子中出现多个语言的情况很常见（比如，I am at 香港站）。

希望可以将其非英文的部分全部转换为英文。

Basic Infos

Student: ZHU Xingye (Joseph)
Supervisor: Prof. Francis C.M. Lau

Dissertation Abstract

Public perception analysis helps improve services and detect issues. This project conducts sentiment analysis and topic labelling task on Hong Kong MTR related tweets under Siemens application scene and compares algorithms adopted in each task. For sentiment analysis, we applies traditional deep neural network such as RNN, CNN on public massive sentiment dataset. For topic labelling, we crawled, labeled, augmented our own dataset and adopts latest transfer learning techniques like BERT, ULMFiT. For both tasks, we use FastText which is a light yet powerful and fast text classification algorithm as baseline.

In our experiments, RNN and ULMFiT achieved the best performance in sentiment analysis and topic labelling task respectively. Our experiments suggest that feature extraction determines model performance while most suitable feature extraction level depends on dataset(size, quality, etc) and label categories. Under extraction like CNN or over extraction like BERT might both lead to worse performance. Also, introducing transfer learning to NLP related tasks in public perception analysis is promising especially when labeled samples are limited.