Skip to content

Commit 7cb475b

Browse files
daachengdaacheng
committed
读取大文件
读取大文件
1 parent 17904a5 commit 7cb475b

File tree

2 files changed

+38
-0
lines changed

2 files changed

+38
-0
lines changed

‎python_basic/Python读取大文本文件.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Python读取大文本文件
2+
* read():一次读取文件的全部内容，可以指定size去读取
3+
* readline():每次读取一行
4+
* readlines():一次读取所有内容并按行返回list
5+
6+
## 按行读取
7+
```python
8+
with open('test.txt', 'r', encoding='utf-8') as fr:
9+
for line in fr:
10+
print(line)
11+
```
12+
13+
## 按块读取
14+
```python
15+
def read_file_chunked(fr, block_size=1024):
16+
while True:
17+
chunk = fr.read(block_size)
18+
if not chunk:
19+
break
20+
yield chunk
21+
22+
23+
if __name__ == '__main__':
24+
with open('test.txt', 'r', encoding='utf-8') as fr:
25+
for chunk in read_file_chunked(fr):
26+
print(chunk)
27+
```
28+
29+
或者
30+
```python
31+
from functools import partial
32+
def read_file_chunked(fr, block_size=1024 * 8):
33+
# 首先使用 partial(fr.read, block_size) 构造一个新的无需参数的函数
34+
# 循环将不断返回 fr.read(block_size) 调用结果，直到其为 '' 时终止
35+
for chunk in iter(partial(fr.read, block_size), ''):
36+
yield chunk
37+
```

‎python_basic/main.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
* [关于tuple](关于tuple.md)
1818
* [json文件处理](json文件处理.md)
1919
* [Excel文件处理](Excel文件处理.md)
20+
* [Python读取大文本文件](Python读取大文本文件.md)
2021
* [Python标准数据类型](Python标准数据类型.md)
2122
* [深拷贝与浅拷贝](深拷贝与浅拷贝.md)
2223
* [匿名函数,列表推导式,字典推导式,生成器表达式,map,filter,reduce](列表推导式.md)

0 commit comments

Comments
 (0)