7310 条评论分享收藏感谢收起weixin.qq.com/r/J3UsNADE612UrURX9yA1 (二维码自动识别)赞同 211 条评论分享收藏感谢收起阿里分析型数据库ads学习记录
阿里分析型数据库ads学习记录
1.ads中非分区表关联问题
无法关联或关联出结果不对，主要因为ads目前的非分区表之间的关联仅支持分区列，且分区数相同，主因为ads关联首先根据分区对应，所以所关联的表与当前表分区字段必须一致，分区数也必须一致，否则得到的结果会跟你想象中差很多。而与分区表关联则不受此限制。使用ads中的维度表
可以与任意表关联，无需对应分区，无需相同分区数。实际上数据量不大的表优先考虑建成维度表，特别是需要与其他表要关联的表。
维度表的劣势：查询性能不随着扩容提升，浪费更多存储空间，但是对于数据量不大的情况下都不是问题。
2.ads不支持带偏移量的limit
实际上无法做成真实分页，而且最多只能取出前1w条数据。可以使用dump方式解决，但是不建议那么做，ads主要关注快速获得查询结果，并不适用此场景。
3.ads支持count(distinct columnName)
需修改后台配置，
/global/config/query --&新增enableUdfSysGroupDistinctConcat bool类型，值false，然后count distinct可以即时生效
4.UNION无法使用或结果错误
目前仅支持分区列UNION，所以带group by的聚合函数结果或查询出的非分区列结果都不可以使用union。
注：以上基于0.8.18版本，后续版本有所更新，会另写文详解。
之后更新会说明版本。
=====更新===========
版本0.9.10
ads性能优化
1.注意过滤条件加上表别名
2.打平关联语句（采用直接关联表而非关联子查询的方式）
3.将关联条件设置为表聚集列
为什么打平（直接关联表而不采用关联子查询的方式）会比子查询快，这里面涉及什么原理？
打平之后，关联的维度表成为右表，join的时候走索引，所以很快；而子查询情况下，维度表就只能作为左表（子查询会自动将其处理为左表），实时表成为右表。实时部分的数据因为没有强大的索引，所以只能走扫描，这样就慢了。
遇到一个坑！！！强调下！！！
select ... a join b where xrsj&....
（a为大表，上百亿数据，b为维度表，上万数据）
这种语句执行的时候，当xrsj这个字段只有a或者b表的时候是完全没问题的，碰巧就是a、b表都有xrsj这个字段的时候，按常理来说，应该报错的！怎么都应该报错才对，然后秀逗了，居然没有报错，还欢快地跑起来了，以至于我一个月都没找到为什么语句那么慢！
所以优化第一句就是过滤条件要加上表别名。
这个bug已经提交给ads开发经理了，不过回复我优先级不会很高，什么时候修复再说。
=====更新===========
分区表与Group-By、Order-By查询
Group-By条件包含分区列，则应该放到第一列，此时结果是精确的且查询性能很好Group-By表达式如果不包含分区列（或第一列不是分区列），则分组数量在范围内（&5000），结果是精确的，但性能随分组数量增加而降低Group-By表达式如果不包含分区列（或第一列不是分区列），且分组数量不在范围内（&=5000），此时结果是不精确的且性能随分组数量增加而降低如果Group-By表达式第一列不是按分区列，则Having语句不支持全局分组TOP(N)：Group-By表达式包含分区列，则应该放到第一列，同时包含Order-By表达式，如果分组数量在范围内（&1000），结果是精确的，但性能随分组数量增加而较大降低全局模糊分组TOP(N)：Group-By表达式第一列如果不是分区列且包含Order-By表达式，而且分组数量不在范围内（&=1000），此时结果是不精确的且性能随分组数量增加而较大降低全数据排序：Order-By表达式第一列如果不是分区列，性能会随排序列或表达式取值增加而较大降使用新引擎不受限制，我估计以后的问题，新引擎都不受限制，毕竟就是为了最大化兼容sql，但是效率较低，最重要的是我现在私有云没有新引擎：)
持续更新中...
【大数据干货】数据进入阿里云数加-分析型数据库AnalyticDB（原ADS）的N种方法
阿里云分析型数据库基本认识
AnalyticDB(原ADS)分区列的最佳实践
3 创建自己的库 ADS.js(第一版本)
FineReport如何用JDBC连接阿里云ADS数据库
阿里云ECS、datahub、RDS、ADS、OTS、MaxCompute及BI报表展示串联
漫谈阿里大数据
powerapps连接阿里云数据库
阿里云流计算产品特点与架构
论文笔记：《Steed:存储树结构数据的分析型数据库》
没有更多推荐了，给你说几个 inline 无法代替宏的地方：&br&&br&1. 循环展开：&br&&br&&div class=&highlight&&&pre&&code class=&language-c&&&span class=&c1&&// loop unroll double&/span&
&span class=&cp&&#define LOOP_UNROLL_DOUBLE(action, actionx2, width) do { \&/span&
&span class=&cp&&
unsigned long __width = (unsigned long)(width); \&/span&
&span class=&cp&&
unsigned long __increment = __width && 2; \&/span&
&span class=&cp&&
for (; __increment & 0; __increment--) { \&/span&
&span class=&cp&&
actionx2; \&/span&
&span class=&cp&&
actionx2; \&/span&
&span class=&cp&&
} \&/span&
&span class=&cp&&
switch (__width & 3) { \&/span&
&span class=&cp&&
case 2: actionx2; \&/span&
&span class=&cp&&
case 3: actionx2; \&/span&
&span class=&cp&&
case 1: \&/span&
&span class=&cp&&
} \&/span&
&span class=&cp&& } while (0)&/span&
&span class=&c1&&// loop unroll quatro&/span&
&span class=&cp&&#define LOOP_UNROLL_QUATRO(action, actionx2, actionx4, width) do { \&/span&
&span class=&cp&&
unsigned long __width = (unsigned long)(width); \&/span&
&span class=&cp&&
unsigned long __increment = __width && 2; \&/span&
&span class=&cp&&
for (; __increment & 0; __increment--) { \&/span&
&span class=&cp&&
actionx4; \&/span&
&span class=&cp&&
} \&/span&
&span class=&cp&&
switch (__width & 3) { \&/span&
&span class=&cp&&
case 2: actionx2; \&/span&
&span class=&cp&&
case 3: actionx2; \&/span&
&span class=&cp&&
case 1: \&/span&
&span class=&cp&&
} \&/span&
&span class=&cp&& } while (0)&/span&
&/code&&/pre&&/div&假设你需要高速循环做一个事情，那么展开循环可以极大的减少CPU分支，并且充分利用CPU流水线的并行效果，比如你开发一个 FIR滤波器来处理信号，那么你的代码如果从 for (...) { .... } 变成循环展开的话，可以这么写：&br&&div class=&highlight&&&pre&&code class=&language-c&&&span class=&n&&LOOP_UNROLL_DOUBLE&/span&&span class=&p&&(&/span&
&span class=&p&&{&/span&
&span class=&n&&x&/span& &span class=&o&&=&/span& &span class=&o&&*&/span&&span class=&n&&src&/span&&span class=&o&&++&/span&&span class=&p&&;&/span&
&span class=&c1&&// do something with x and h and output to y&/span&
&span class=&o&&*&/span&&span class=&n&&dst&/span&&span class=&o&&++&/span& &span class=&o&&=&/span& &span class=&n&&y&/span&&span class=&p&&;&/span&
&span class=&p&&},&/span&
&span class=&p&&{&/span&
&span class=&n&&x1&/span& &span class=&o&&=&/span& &span class=&o&&*&/span&&span class=&n&&src&/span&&span class=&o&&++&/span&&span class=&p&&;&/span&
&span class=&n&&x2&/span& &span class=&o&&=&/span& &span class=&o&&*&/span&&span class=&n&&src&/span&&span class=&o&&++&/span&&span class=&p&&;&/span&
&span class=&c1&&// do something with x1 and h and output to y1&/span&
&span class=&c1&&// do something with x2 and h and output to y2&/span&
&span class=&o&&*&/span&&span class=&n&&dst&/span&&span class=&o&&++&/span& &span class=&o&&=&/span& &span class=&n&&y1&/span&&span class=&p&&;&/span&
&span class=&o&&*&/span&&span class=&n&&dst&/span&&span class=&o&&++&/span& &span class=&o&&=&/span& &span class=&n&&y2&/span&&span class=&p&&;&/span&
&span class=&p&&},&/span&
&span class=&n&&nsamples&/span&&span class=&p&&,&/span&
&span class=&p&&);&/span&
&/code&&/pre&&/div&如此写法将每个循环只计算一个 sample，变为每个循环同时计算两个sample，分开写代码，也能更好的利用 SIMD去加速同时多个 sample的计算过程，这就是利用循环展开来优化性能的用法，直接传 &{...}& 里面的运行代码给宏，宏不变，但是每处使用LOOP_UNROLL的地方 &{.. } & 中的代码都是不同的，inline是代替不了的，你总不至于传个函数指针过去吧，这时性能优化方面情况。&br&&br&2. 函数组装：&br&&br&想象一下，你写图形图像的代码，现在你需要给像素合成实现 SRC_ATOP， SRC_OVER, SRC_IN, SRC_OUT, DST_ATOP, DST_OVER, DST_IN, DST_OUT, XOR, PLUS, ALLANON, TINT, DIFF, DARKEN, LIGHTEN, SCREEN, OVERLAY 等等 二十种像素合成的方法，你如果不用宏，那么你需要写多少个函数？20多个看起来类似的函数，你不得写疯了么？此时用函数指针其实是很浪费性能的事情，那么该如何写呢？你可以规定一系列用来计算composite的方法，接受两组 RGBA，生成新的，比如：&br&&div class=&highlight&&&pre&&code class=&language-c&&&span class=&cm&&/* compositing */&/span&
&span class=&cp&&#define IBLEND_COMPOSITE(sr, sg, sb, sa, dr, dg, db, da, FS, FD) do { \&/span&
&span class=&cp&&
(dr) = _ipixel_mullut[(FS)][(sr)] + _ipixel_mullut[(FD)][(dr)]; \&/span&
&span class=&cp&&
(dg) = _ipixel_mullut[(FS)][(sg)] + _ipixel_mullut[(FD)][(dg)]; \&/span&
&span class=&cp&&
(db) = _ipixel_mullut[(FS)][(sb)] + _ipixel_mullut[(FD)][(db)]; \&/span&
&span class=&cp&&
(da) = _ipixel_mullut[(FS)][(sa)] + _ipixel_mullut[(FD)][(da)]; \&/span&
&span class=&cp&& } while (0)&/span&
&span class=&cm&&/* premultiply: src over */&/span&
&span class=&cp&&#define IBLEND_OP_SRC_OVER(sr, sg, sb, sa, dr, dg, db, da) do { \&/span&
&span class=&cp&&
IUINT32 FD = 255 - (sa); \&/span&
&span class=&cp&&
IBLEND_COMPOSITE(sr, sg, sb, sa, dr, dg, db, da, 255, FD); \&/span&
&span class=&cp&& } while (0)&/span&
&span class=&cm&&/* premultiply: dst atop */&/span&
&span class=&cp&&#define IBLEND_OP_DST_ATOP(sr, sg, sb, sa, dr, dg, db, da) do { \&/span&
&span class=&cp&&
IUINT32 FS = 255 - (da); \&/span&
&span class=&cp&&
IUINT32 FD = (sa); \&/span&
&span class=&cp&&
IBLEND_COMPOSITE(sr, sg, sb, sa, dr, dg, db, da, FS, FD); \&/span&
&span class=&cp&& } while (0)&/span&
&span class=&cm&&/* premultiply: dst in */&/span&
&span class=&cp&&#define IBLEND_OP_DST_IN(sr, sg, sb, sa, dr, dg, db, da) do { \&/span&
&span class=&cp&&
IUINT32 FD = (sa); \&/span&
&span class=&cp&&
IBLEND_COMPOSITE(sr, sg, sb, sa, dr, dg, db, da, 0, FD); \&/span&
&span class=&cp&& } while (0)&/span&
&/code&&/pre&&/div&然后用 #连接各种方法和格式，生成不同的函数，比如：&br&&div class=&highlight&&&pre&&code class=&language-c&&&span class=&cp&&#define IPIXEL_COMPOSITE_FN(name, opname) \&/span&
&span class=&cp&&static void ipixel_comp_##name(IUINT32 *dst, const IUINT32 *src, int w)\&/span&
&span class=&cp&&{ \&/span&
&span class=&cp&& IUINT32 sr, sg, sb, sa, dr, dg, db, \&/span&
&span class=&cp&& for (; w & 0; dst++, src++, w--) { \&/span&
&span class=&cp&&
_ipixel_load_card(src, sr, sg, sb, sa); \&/span&
&span class=&cp&&
_ipixel_load_card(dst, dr, dg, db, da); \&/span&
&span class=&cp&&
IBLEND_OP_##opname(sr, sg, sb, sa, dr, dg, db, da); \&/span&
&span class=&cp&&
dst[0] = IRGBA_TO_A8R8G8B8(dr, dg, db, da); \&/span&
&span class=&cp&& } \&/span&
&span class=&cp&&}&/span&
&/code&&/pre&&/div&然后开始生成我们的各种合成函数：&br&&div class=&highlight&&&pre&&code class=&language-c&&&span class=&n&&IPIXEL_COMPOSITE_PREMUL&/span&&span class=&p&&(&/span&&span class=&n&&pre_xor&/span&&span class=&p&&,&/span& &span class=&n&&XOR&/span&&span class=&p&&);&/span&
&span class=&n&&IPIXEL_COMPOSITE_PREMUL&/span&&span class=&p&&(&/span&&span class=&n&&pre_plus&/span&&span class=&p&&,&/span& &span class=&n&&PLUS&/span&&span class=&p&&);&/span&
&span class=&n&&IPIXEL_COMPOSITE_PREMUL&/span&&span class=&p&&(&/span&&span class=&n&&pre_src_atop&/span&&span class=&p&&,&/span& &span class=&n&&SRC_ATOP&/span&&span class=&p&&);&/span&
&span class=&n&&IPIXEL_COMPOSITE_PREMUL&/span&&span class=&p&&(&/span&&span class=&n&&pre_src_in&/span&&span class=&p&&,&/span& &span class=&n&&SRC_IN&/span&&span class=&p&&);&/span&
&span class=&n&&IPIXEL_COMPOSITE_PREMUL&/span&&span class=&p&&(&/span&&span class=&n&&pre_src_out&/span&&span class=&p&&,&/span& &span class=&n&&SRC_OUT&/span&&span class=&p&&);&/span&
&span class=&n&&IPIXEL_COMPOSITE_PREMUL&/span&&span class=&p&&(&/span&&span class=&n&&pre_src_over&/span&&span class=&p&&,&/span& &span class=&n&&SRC_OVER&/span&&span class=&p&&);&/span&
&span class=&n&&IPIXEL_COMPOSITE_PREMUL&/span&&span class=&p&&(&/span&&span class=&n&&pre_dst_atop&/span&&span class=&p&&,&/span& &span class=&n&&DST_ATOP&/span&&span class=&p&&);&/span&
&span class=&n&&IPIXEL_COMPOSITE_PREMUL&/span&&span class=&p&&(&/span&&span class=&n&&pre_dst_in&/span&&span class=&p&&,&/span& &span class=&n&&DST_IN&/span&&span class=&p&&);&/span&
&span class=&n&&IPIXEL_COMPOSITE_PREMUL&/span&&span class=&p&&(&/span&&span class=&n&&pre_dst_out&/span&&span class=&p&&,&/span& &span class=&n&&DST_OUT&/span&&span class=&p&&);&/span&
&span class=&n&&IPIXEL_COMPOSITE_PREMUL&/span&&span class=&p&&(&/span&&span class=&n&&pre_dst_over&/span&&span class=&p&&,&/span& &span class=&n&&DST_OVER&/span&&span class=&p&&);&/span&
&/code&&/pre&&/div&这样你相当于定义了：&br&&div class=&highlight&&&pre&&code class=&language-text&&ipixel_comp_pre_xor (...)
ipixel_comp_pre_plus (...)
ipixel_comp_dst_over (...)
&/code&&/pre&&/div&等好几个函数了，并且这些函数都是被你 “组装” 出来的，你并没有使用函数指针，也没有笨重的去写20多个函数。进一步如果你写图形图像你会发现你需要面对多种设备的像素格式，从 A8R8G8B8, A8B8G8R8 到 A1R5G5B5 , 主流需要处理的像素格式都有10多种。&br&&br&那么你可以把 “从不同格式读取 r,g,b,a”， 以及 “将 r,g,b,a组装成任意格式”，展开成很多个宏，然后不管你在这些像素格式里面做转换还是要做一些其他处理，你都可以用任意的 “像素读写” 宏 + “像素计算” 宏 组装成一个个具体需要的函数。&br&&br&所以用宏来解决性能问题，并且简化自己的程序设计往往能起到 inline不能起的作用，甚至能完成很多 template 所不能完成的任务。&br&&br&3. 数据结构和算法：&br&&br&具体可以参考 Linux Kernel的 include/linux/list.h:&br&&div class=&highlight&&&pre&&code class=&language-c&&&span class=&k&&struct&/span& &span class=&n&&list_head&/span& &span class=&p&&{&/span&
&span class=&k&&struct&/span& &span class=&n&&list_head&/span& &span class=&o&&*&/span&&span class=&n&&next&/span&&span class=&p&&,&/span& &span class=&o&&*&/span&&span class=&n&&prev&/span&&span class=&p&&;&/span&
&span class=&p&&};&/span&
&span class=&cp&&#define INIT_LIST_HEAD(ptr) do { \&/span&
&span class=&cp&& (ptr)-&next = (ptr); (ptr)-&prev = (ptr); \&/span&
&span class=&cp&&} while (0)&/span&
&span class=&cm&&/*&/span&
&span class=&cm&& * Insert a new entry between two known consecutive entries. &/span&
&span class=&cm&& *&/span&
&span class=&cm&& * This is only for internal list manipulation where we know&/span&
&span class=&cm&& * the prev/next entries already!&/span&
&span class=&cm&& */&/span&
&span class=&k&&static&/span& &span class=&n&&__inline__&/span& &span class=&kt&&void&/span& &span class=&nf&&__list_add&/span&&span class=&p&&(&/span&&span class=&k&&struct&/span& &span class=&n&&list_head&/span& &span class=&o&&*&/span& &span class=&n&&new&/span&&span class=&p&&,&/span&
&span class=&k&&struct&/span& &span class=&n&&list_head&/span& &span class=&o&&*&/span& &span class=&n&&prev&/span&&span class=&p&&,&/span&
&span class=&k&&struct&/span& &span class=&n&&list_head&/span& &span class=&o&&*&/span& &span class=&n&&next&/span&&span class=&p&&)&/span&
&span class=&p&&{&/span&
&span class=&n&&next&/span&&span class=&o&&-&&/span&&span class=&n&&prev&/span& &span class=&o&&=&/span& &span class=&n&&new&/span&&span class=&p&&;&/span&
&span class=&n&&new&/span&&span class=&o&&-&&/span&&span class=&n&&next&/span& &span class=&o&&=&/span& &span class=&n&&next&/span&&span class=&p&&;&/span&
&span class=&n&&new&/span&&span class=&o&&-&&/span&&span class=&n&&prev&/span& &span class=&o&&=&/span& &span class=&n&&prev&/span&&span class=&p&&;&/span&
&span class=&n&&prev&/span&&span class=&o&&-&&/span&&span class=&n&&next&/span& &span class=&o&&=&/span& &span class=&n&&new&/span&&span class=&p&&;&/span&
&span class=&p&&}&/span&
&/code&&/pre&&/div&这里定义了一个 LIST，kernel中，能用 inline的地方都用了，但是有些地方用不了，比如，你有一个结构体 （netfilter 部分）：&br&&div class=&highlight&&&pre&&code class=&language-c&&&span class=&k&&struct&/span& &span class=&n&&nf_hook_ops&/span&
&span class=&p&&{&/span&
&span class=&k&&struct&/span& &span class=&n&&list_head&/span& &span class=&n&&list&/span&&span class=&p&&;&/span&
&span class=&cm&&/* User fills in from here down. */&/span&
&span class=&n&&nf_hookfn&/span& &span class=&o&&*&/span&&span class=&n&&hook&/span&&span class=&p&&;&/span&
&span class=&kt&&int&/span& &span class=&n&&pf&/span&&span class=&p&&;&/span&
&span class=&kt&&int&/span& &span class=&n&&hooknum&/span&&span class=&p&&;&/span&
&span class=&cm&&/* Hooks are ordered in ascending priority. */&/span&
&span class=&kt&&int&/span& &span class=&n&&priority&/span&&span class=&p&&;&/span&
&span class=&p&&};&/span&
&/code&&/pre&&/div&然后你有一个链表，记录着很多 nf_hook_ops，你取到了其中一个节点的指针，其实是指向结构体的 &list这个成员的，你需要得到对应结构体的指针，那么你可以用下面的 list 的宏：&br&&div class=&highlight&&&pre&&code class=&language-c&&&span class=&cm&&/**&/span&
&span class=&cm&& * list_entry - get the struct for this entry&/span&
&span class=&cm&& * @ptr: the &struct list_head pointer.&/span&
&span class=&cm&& * @type: the type of the struct this is embedded in.&/span&
&span class=&cm&& * @member: the name of the list_struct within the struct.&/span&
&span class=&cm&& */&/span&
&span class=&cp&&#define list_entry(ptr, type, member) \&/span&
&span class=&cp&& ((type *)((char *)(ptr)-(unsigned long)(&((type *)0)-&member)))&/span&
&/code&&/pre&&/div&比如，list_entry(ptr, struct nf_hook_ops, list) 就能根据节点指针，和在某个 struct里面的位置，取到某个节点对应的 struct的指针了。这个做法，用 inline也是没法做的。&br&&br&同样的应用，在 Kernel中，还有红黑树 rbtree.h，rbtree.c中的实现，和 list很类似，大量的宏应用。Linux 用基础的宏实现的 list, rbtree等基础数据结构，用起来是相当方便的，有些地方比 std::list, std::map 都方便多了，比 STL性能高的同时，避免象模版一样为每种类型生成不同的代码，让你的二进制程序变得很臃肿。&br&&br&比如你在做题的时候，用上了这样的数据结构，你程序会比用 stl容器的代码更高效和精简，同时你不知道目标平台 STL是怎么实现的，你无法控制，明明我在这个平台写着很快的代码，为何换个平台又慢了，为了追求究极性能，这样重新定义数据结构，其实是可以理解的。&br&&br&4. 其他 inline 无法代替宏的地方&br&&ul&&li&针对不同平台特性（比如整数是32还是64，lsb还是msb）写出的优化代码。&br&&/li&&li&泛型的模拟&br&&/li&&li&小型高频重复的代码片&br&&/li&&li&硬件操作的定义&/li&&/ul&等等，很多情况，inline或者 template还是无法把宏给代替了，所以很多开源项目的代码里面，大量的出现各种宏，主要是出于这些方面的考虑。&br&&br&-----------
给你说几个 inline 无法代替宏的地方： 1. 循环展开： // loop unroll double
#define LOOP_UNROLL_DOUBLE(action, actionx2, width) do { \
unsigned long __width = (unsigned long)(width); \
unsigned long __increment = __width && 2; \
&figure&&img src=&https://pic1.zhimg.com/v2-47fe2d7f754bfc247d95bdab959c8acb_b.jpg& data-rawwidth=&804& data-rawheight=&362& class=&origin_image zh-lightbox-thumb& width=&804& data-original=&https://pic1.zhimg.com/v2-47fe2d7f754bfc247d95bdab959c8acb_r.jpg&&&/figure&既然播放控制函数和界面渲染我们都弄出来了，那么能把两者串起来的就是交互功能模块了！一旦交互函数弄好了，整个播放器也就大功告成了。这一期就一气呵成，记录下落网播放器最后的收尾工作！&p&在急于写代码之前，我们先明确一下交互功能，我的设计如下，w/s键为上下选择歌曲，p键为播放暂停，-/+键为音量控制！&/p&&p&shell脚本的读入键盘大家应该知道就是read，但是read通常需要额外按一次回车来确认。有没有一次只读取一个字节且免回车的方式呢？有的那就是带上参数的读——&read -n 1&。当然你直接在终端尝试应该看不出效果。可以写一个test.sh脚本来测试。&/p&&div class=&highlight&&&pre&&code class=&language-bash&&&span&&/span&&span class=&ch&&#!/bin/bash&/span&
&span class=&k&&function&/span& keymap &span class=&o&&{&/span&
&span class=&c1&&#-s参数silent，免回显，否则会在终端上显示输入&/span&
&span class=&nb&&read&/span& -sn &span class=&m&&1&/span& key
&span class=&nb&&echo&/span& &span class=&s2&&&you just push &/span&&span class=&nv&&$key&/span&&span class=&s2&&&&/span&
&span class=&o&&}&/span&
&span class=&k&&while&/span& true&span class=&p&&;&/span& &span class=&k&&do&/span&
&span class=&k&&done&/span&
&/code&&/pre&&/div&效果如下&p&&figure&&img src=&https://pic2.zhimg.com/v2-0ce526de0ea87b56490dcf_b.jpg& data-rawwidth=&772& data-rawheight=&240& class=&origin_image zh-lightbox-thumb& width=&772& data-original=&https://pic2.zhimg.com/v2-0ce526de0ea87b56490dcf_r.jpg&&&/figure&于是改造luoo.sh，添加如下函数&/p&&div class=&highlight&&&pre&&code class=&language-bash&&&span&&/span&&span class=&nb&&declare&/span& &span class=&nv&&CONTROL_FILE&/span&&span class=&o&&=&/span&/tmp/contorl
. ./control.sh &span class=&c1&&#加载播放库&/span&
&span class=&k&&function&/span& choose &span class=&o&&{&/span&
&span class=&k&&if&/span& &span class=&o&&[&/span& &span class=&nv&&$1&/span& &span class=&o&&=&/span& next &span class=&o&&]&/span& &span class=&p&&;&/span&&span class=&k&&then&/span&
&span class=&nb&&let&/span& &span class=&nv&&cursor&/span&&span class=&o&&=(&/span&&span class=&nv&&$cursor&/span&+1&span class=&o&&)&/span&%&span class=&nv&&$size&/span&
&span class=&k&&elif&/span& &span class=&o&&[&/span& &span class=&nv&&$1&/span& &span class=&o&&=&/span& prev &span class=&o&&]&/span&&span class=&p&&;&/span& &span class=&k&&then&/span&
&span class=&nb&&let&/span& &span class=&nv&&move&/span&&span class=&o&&=&/span&&span class=&nv&&$cursor&/span&-1
&span class=&nb&&let&/span& &span class=&nv&&cursor&/span&&span class=&o&&=&/span&&span class=&nv&&$move&/span&&span class=&o&&==&/span&-1?&span class=&nv&&$size&/span&-1:&span class=&nv&&$move&/span&
&span class=&k&&fi&/span&
&span class=&o&&}&/span&
&span class=&k&&function&/span& enjoy &span class=&o&&{&/span&
&span class=&c1&&#cursor由于是0开始的，但是乐曲是从01开始的，所以是cursor+1&/span&
&span class=&nb&&local&/span& &span class=&nv&&url&/span&&span class=&o&&=&/span&&span class=&sb&&`&/span&&span class=&nb&&printf&/span& http://mp3-cdn.luoo.net/low/luoo/radio%s/%02d.mp3 &span class=&se&&\&/span&
&span class=&nv&&$vol&/span& $&span class=&o&&[&/span&cursor+1&span class=&o&&]&/span&&span class=&sb&&`&/span& & /dev/null
&span class=&k&&if&/span& &span class=&o&&[&/span& ! &span class=&nv&&$playing&/span& -eq &span class=&nv&&$cursor&/span& &span class=&o&&]&/span& &span class=&p&&;&/span&&span class=&k&&then&/span&
&span class=&c1&&#play是一个阻塞调用，末尾加上&，变成非阻塞&/span&
play &span class=&nv&&$url&/span& &span class=&p&&&&/span&
&span class=&k&&else&/span&
&span class=&k&&fi&/span&
&span class=&o&&}&/span&
&span class=&k&&function&/span& keymap &span class=&o&&{&/span&
&span class=&nb&&read&/span& -sn &span class=&m&&1&/span& key
&span class=&k&&case&/span& &span class=&nv&&$key&/span& in
&span class=&s1&&'w'&/span&&span class=&o&&)&/span& choose prev &span class=&p&&;;&/span&
&span class=&s1&&'s'&/span&&span class=&o&&)&/span& choose next &span class=&p&&;;&/span&
&span class=&s1&&'p'&/span&&span class=&o&&)&/span& enjoy &span class=&p&&;;&/span&
&span class=&k&&esac&/span&
&span class=&o&&}&/span&
&span class=&c1&&#最后改造一下main函数，变成死循环&/span&
tput smcup
&span class=&nb&&echo&/span& -e &span class=&nv&&$NORMAL&/span&
loadsongs &span class=&nv&&$1&/span&
&span class=&k&&if&/span& &span class=&o&&[&/span& -p &span class=&nv&&$CONTROL_FILE&/span& &span class=&o&&]&/span& &span class=&p&&;&/span&&span class=&k&&then&/span&
rm &span class=&nv&&$CONTROL_FILE&/span&
&span class=&k&&fi&/span&
mkfifo &span class=&nv&&$CONTROL_FILE&/span&
&span class=&k&&while&/span& &span class=&nb&&true&/span& &span class=&p&&;&/span&&span class=&k&&do&/span&
&span class=&k&&done&/span&
tput rmcup
&/code&&/pre&&/div&好了，再执行的时候，我相信已经可以自由上下选择了并播放、暂停音乐了。很简单吧？&p&好如果你只做到这一步，基本上达到了我最开始写文章时候的完成度，80%吧。&/p&&p&首先是播放函数除了play函数其他函数都没有考虑到mplayer进程不存在的情况，如果消费者不存在，一上来就往管道写内容，比如我一进来谁都没有选定，直接按了p键，就会阻塞。整个软件也就卡住了。为了健壮，cursor不要默认为-1应为0&br&&/p&&p&然后如果你能注意到我的播放器在播放的选项之前是有一个playing的状态的。这个状态就是一个数字变量，但是，这个变量luoo.sh和control.sh同时都要用到，shell的特殊就在于变量只能由调用者向被调用者传递，而不能反过来，这个传递呢，其实是复制了一份拷贝进被调用者，也就是说control.sh即便对playing变量修改是不会对luoo.sh造成影响，要解决这个问题呢，得考虑使用文件。control.sh和luoo.sh同时读写/tmp/playing这个文件，就可以实现共享。&/p&&p&于是先改造control.sh&/p&&div class=&highlight&&&pre&&code class=&language-bash&&&span&&/span&&span class=&ch&&#!/bin/bash&/span&
&span class=&nb&&declare&/span& -x CONTROL_FILE
&span class=&c1&&#-x为导出变量，全局变量&/span&
&span class=&nb&&declare&/span& -x PLAYING_FILE
&span class=&c1&&#playing共享变量文件&/span&
&span class=&k&&function&/span& mplayerpause &span class=&o&&{&/span&
&span class=&nb&&local&/span& &span class=&nv&&playerpid&/span&&span class=&o&&=&/span&&span class=&sb&&`&/span&ps -ef&span class=&p&&|&/span&grep mplayer &span class=&se&&\&/span&
&span class=&p&&|&/span&grep -v grep &span class=&se&&\&/span&
&span class=&p&&|&/span&awk &span class=&s1&&'{print $2}'&/span& &span class=&se&&\&/span&
&span class=&p&&|&/span&head -n1&span class=&sb&&`&/span&
&span class=&k&&if&/span& &span class=&o&&[&/span& -n &span class=&s2&&&&/span&&span class=&nv&&$playerpid&/span&&span class=&s2&&&&/span& &span class=&o&&]&/span& &span class=&p&&;&/span&&span class=&k&&then&/span&
&span class=&nb&&echo&/span& pause & &span class=&nv&&$CONTROL_FILE&/span&
&span class=&k&&fi&/span&
&span class=&o&&}&/span&
&span class=&k&&function&/span& mplayerquit &span class=&o&&{&/span&
&span class=&nb&&local&/span& &span class=&nv&&playerpid&/span&&span class=&o&&=&/span&&span class=&sb&&`&/span&ps -ef&span class=&p&&|&/span&grep mplayer &span class=&se&&\&/span&
&span class=&p&&|&/span&grep -v grep &span class=&se&&\&/span&
&span class=&p&&|&/span&awk &span class=&s1&&'{print $2}'&/span& &span class=&se&&\&/span&
&span class=&p&&|&/span&head -n1&span class=&sb&&`&/span&
&span class=&k&&if&/span& &span class=&o&&[&/span& -n &span class=&s2&&&&/span&&span class=&nv&&$playerpid&/span&&span class=&s2&&&&/span& &span class=&o&&]&/span& &span class=&p&&;&/span&&span class=&k&&then&/span&
&span class=&nb&&echo&/span& quit & &span class=&nv&&$CONTROL_FILE&/span&
&span class=&k&&fi&/span&
&span class=&o&&}&/span&
&span class=&k&&function&/span& mplayerplay &span class=&o&&{&/span&
&span class=&c1&&#接受参数$1为音乐地址&/span&
&span class=&nb&&local&/span& &span class=&nv&&playerpid&/span&&span class=&o&&=&/span&&span class=&sb&&`&/span&ps -ef&span class=&p&&|&/span&grep mplayer &span class=&se&&\&/span&
&span class=&p&&|&/span&grep -v grep &span class=&se&&\&/span&
&span class=&p&&|&/span&awk &span class=&s1&&'{print $2}'&/span& &span class=&se&&\&/span&
&span class=&p&&|&/span&head -n1&span class=&sb&&`&/span&
&span class=&k&&if&/span& &span class=&o&&[&/span& -z &span class=&s2&&&&/span&&span class=&nv&&$playerpid&/span&&span class=&s2&&&&/span& &span class=&o&&]&/span& &span class=&p&&;&/span&&span class=&k&&then&/span&
&span class=&c1&&#检查mplayer进程是否存在，如果不存在则启动mplayer&/span&
&span class=&nb&&echo&/span& &span class=&nv&&$CONTROL_FILE&/span& && luoo.log
mplayer -input &span class=&nv&&file&/span&&span class=&o&&=&/span&&span class=&nv&&$CONTROL_FILE&/span& &span class=&nv&&$1&/span& & /dev/null 2&&span class=&p&&&&/span&1
&span class=&nb&&echo&/span& -1 & &span class=&nv&&$PLAYING_FILE&/span&
&span class=&c1&&#播放结束需修改“当前播放”&/span&
&span class=&k&&else&/span&
&span class=&c1&&#如果存在，则写入操作指令到控制文件&/span&
&span class=&nb&&echo&/span& &span class=&s2&&&load &/span&&span class=&nv&&$1&/span&&span class=&s2&&&&/span& & &span class=&nv&&$CONTROL_FILE&/span&
&span class=&k&&fi&/span&
&span class=&o&&}&/span&
&/code&&/pre&&/div&接下来，luoo.sh的最终形态！&div class=&highlight&&&pre&&code class=&language-bash&&&span&&/span&&span class=&ch&&#!/bin/bash&/span&
&span class=&nb&&declare&/span& -a songs &span class=&c1&&#-a定义songs为Array结构&/span&
&span class=&nb&&declare&/span& -i &span class=&nv&&size&/span&&span class=&o&&=&/span&&span class=&m&&0&/span& &span class=&c1&&#-i定义size为数字类型 songs的长度&/span&
&span class=&nb&&declare&/span& -i &span class=&nv&&cursor&/span&&span class=&o&&=&/span&&span class=&m&&0&/span& &span class=&c1&&#定义选中光标&/span&
&span class=&nb&&declare&/span& vol &span class=&c1&&#什么参数都不加，默认为字符串&/span&
&span class=&nb&&declare&/span& title
&span class=&nb&&declare&/span& &span class=&nv&&CONTROL_FILE&/span&&span class=&o&&=&/span&/tmp/contorl
&span class=&nb&&declare&/span& &span class=&nv&&PLAYING_FILE&/span&&span class=&o&&=&/span&/tmp/playing
. ./control.sh
&span class=&nv&&SELECTED&/span&&span class=&o&&=&/span&&span class=&s2&&&\033[31;44m&&/span&
&span class=&c1&&#选中的时候前景色为红，背景色青蓝&/span&
&span class=&nv&&NORMAL&/span&&span class=&o&&=&/span&&span class=&s2&&&\033[37;44m&&/span&
&span class=&c1&&#非选中请景色为白，背景色仍是青蓝&/span&
&span class=&k&&function&/span& debug &span class=&o&&{&/span&
&span class=&c1&&#日志功能，带上时间点打印到luoo.log文件中去，方便调试&/span&
&span class=&nb&&echo&/span& &span class=&sb&&`&/span&date +%T:&span class=&sb&&`&/span& &span class=&nv&&$@&/span& && luoo.log
&span class=&o&&}&/span&
&span class=&c1&&#loadsongs(vol),加载某一期的歌曲专辑&/span&
&span class=&k&&function&/span& loadsongs &span class=&o&&{&/span&
debug &span class=&s2&&&loadsongs &/span&&span class=&nv&&$1&/span&&span class=&s2&&&&/span&
&span class=&nv&&vol&/span&&span class=&o&&=&/span&&span class=&nv&&$1&/span&
&span class=&nb&&local&/span& &span class=&nv&&html&/span&&span class=&o&&=&/span&&span class=&k&&$(&/span&curl -s http://www.luoo.net/music/&span class=&nv&&$1&/span&&span class=&k&&)&/span&
&span class=&nv&&title&/span&&span class=&o&&=&/span&&span class=&k&&$(&/span&&span class=&nb&&echo&/span& &span class=&nv&&$html&/span& &span class=&se&&\&/span&
&span class=&p&&|&/span&grep -oP &span class=&s2&&&vol-title\&&[^&]+&&&/span& &span class=&se&&\&/span&
&span class=&p&&|&/span&grep -oP &span class=&s2&&&&[^&]+&&&/span&&span class=&k&&)&/span&
&span class=&nv&&title&/span&&span class=&o&&=&/span&&span class=&si&&${&/span&&span class=&nv&&title&/span&&span class=&p&&:&/span&&span class=&nv&&1&/span&&span class=&k&&:-&/span&&span class=&nv&&1&/span&&span class=&si&&}&/span&
&span class=&nb&&local&/span& &span class=&nv&&body&/span&&span class=&o&&=&/span&&span class=&k&&$(&/span&&span class=&nb&&echo&/span& &span class=&nv&&$html&/span& &span class=&se&&\&/span&
&span class=&p&&|&/span&grep -oP &span class=&s2&&&trackname btn-play\&&[^&]+&&&/span& &span class=&se&&\&/span&
&span class=&p&&|&/span&grep -oP &span class=&s2&&&&[^&]+&&&/span&&span class=&k&&)&/span&
&span class=&c1&&#read -r 意思是每次读一行！否则每次读一个字符串，读到line变量中！&/span&
&span class=&k&&while&/span& &span class=&nb&&read&/span& -r line&span class=&p&&;&/span&
&span class=&k&&do&/span&
songs&span class=&o&&[&/span&&span class=&nv&&$size&/span&&span class=&o&&]=&/span&&span class=&si&&${&/span&&span class=&nv&&line&/span&&span class=&p&&:&/span&&span class=&nv&&1&/span&&span class=&k&&:-&/span&&span class=&nv&&1&/span&&span class=&si&&}&/span&
&span class=&k&&$((&/span&size++&span class=&k&&))&/span&
&span class=&k&&done&/span&&span class=&s&&&&EOF&/span&
&span class=&s&&
$body&/span&
&span class=&s&&EOF&/span&
&span class=&o&&}&/span&
&span class=&k&&function&/span& draw &span class=&o&&{&/span&
&span class=&nb&&read&/span& playing & &span class=&nv&&$PLAYING_FILE&/span&
cat &span class=&s&&&&EOF&/span&
&span class=&s&&
&vol.$vol&$title&/span&
&span class=&s&&EOF&/span&
&span class=&k&&for&/span& index in &span class=&si&&${&/span&&span class=&p&&!songs[@]&/span&&span class=&si&&}&/span& &span class=&p&&;&/span&&span class=&k&&do&/span&
&span class=&nb&&printf&/span& &span class=&s2&&&\t&&/span&
&span class=&k&&if&/span& &span class=&o&&[&/span& &span class=&nv&&$playing&/span& -eq &span class=&nv&&$index&/span& &span class=&o&&]&/span& &span class=&p&&;&/span&&span class=&k&&then&/span&
&span class=&nb&&printf&/span& &span class=&s2&&&[&&/span&&span class=&nv&&$BOLD&/span&&span class=&s2&&&playing&&/span&&span class=&nv&&$NORMAL&/span&&span class=&s2&&&]&&/span&
&span class=&k&&else&/span&
&span class=&nb&&printf&/span& &span class=&s2&&&[-------]&&/span&
&span class=&k&&fi&/span&
&span class=&k&&if&/span& &span class=&o&&[&/span& &span class=&nv&&$cursor&/span& -eq &span class=&nv&&$index&/span& &span class=&o&&]&/span& &span class=&p&&;&/span&&span class=&k&&then&/span&
&span class=&nb&&echo&/span& -e &span class=&nv&&$SELECTED&/span&&span class=&si&&${&/span&&span class=&nv&&songs&/span&&span class=&p&&[&/span&&span class=&nv&&$index&/span&&span class=&p&&]&/span&&span class=&si&&}&/span&&span class=&nv&&$NORMAL&/span&
&span class=&k&&else&/span&
&span class=&nb&&echo&/span& -e &span class=&nv&&$NORMAL&/span&&span class=&si&&${&/span&&span class=&nv&&songs&/span&&span class=&p&&[&/span&&span class=&nv&&$index&/span&&span class=&p&&]&/span&&span class=&si&&}&/span&&span class=&nv&&$NORMAL&/span&
&span class=&k&&fi&/span&
&span class=&k&&done&/span&
&span class=&o&&}&/span&
&span class=&k&&function&/span& keymap &span class=&o&&{&/span&
&span class=&nb&&read&/span& -sn &span class=&m&&1&/span& key
&span class=&k&&case&/span& &span class=&nv&&$key&/span& in
&span class=&s1&&'s'&/span&&span class=&o&&)&/span& choose next &span class=&p&&;;&/span&
&span class=&s1&&'w'&/span&&span class=&o&&)&/span& choose prev &span class=&p&&;;&/span&
&span class=&s1&&'p'&/span&&span class=&o&&)&/span& enjoy &span class=&p&&;;&/span&
&span class=&k&&esac&/span&
&span class=&o&&}&/span&
&span class=&k&&function&/span& choose &span class=&o&&{&/span&
&span class=&k&&if&/span& &span class=&o&&[&/span& &span class=&nv&&$1&/span& &span class=&o&&=&/span& next &span class=&o&&]&/span& &span class=&p&&;&/span&&span class=&k&&then&/span&
&span class=&nb&&let&/span& &span class=&nv&&cursor&/span&&span class=&o&&=(&/span&&span class=&nv&&$cursor&/span&+1&span class=&o&&)&/span&%&span class=&nv&&$size&/span&
&span class=&k&&elif&/span& &span class=&o&&[&/span& &span class=&nv&&$1&/span& &span class=&o&&=&/span& prev &span class=&o&&]&/span&&span class=&p&&;&/span& &span class=&k&&then&/span&
&span class=&nb&&let&/span& &span class=&nv&&move&/span&&span class=&o&&=&/span&&span class=&nv&&$cursor&/span&-1
&span class=&nb&&let&/span& &span class=&nv&&cursor&/span&&span class=&o&&=&/span&&span class=&nv&&$move&/span&&span class=&o&&==&/span&-1?&span class=&nv&&$size&/span&-1:&span class=&nv&&$move&/span&
&span class=&k&&fi&/span&
&span class=&o&&}&/span&
&span class=&k&&function&/span& enjoy &span class=&o&&{&/span&
&span class=&nb&&read&/span& playing & &span class=&nv&&$PLAYING_FILE&/span&
&span class=&nb&&local&/span& &span class=&nv&&url&/span&&span class=&o&&=&/span&&span class=&sb&&`&/span&&span class=&nb&&printf&/span& http://mp3-cdn.luoo.net/low/luoo/radio%s/%02d.mp3 &span class=&se&&\&/span&
&span class=&nv&&$vol&/span& $&span class=&o&&[&/span&cursor+1&span class=&o&&]&/span&&span class=&sb&&`&/span& & /dev/null
&span class=&k&&if&/span& &span class=&o&&[&/span& ! &span class=&nv&&$playing&/span& -eq &span class=&nv&&$cursor&/span& &span class=&o&&]&/span& &span class=&p&&;&/span&&span class=&k&&then&/span&
&span class=&c1&&#播放结束后需要触发一次重绘所以在play之后紧接着draw&/span&
&span class=&o&&(&/span&mplayerplay &span class=&nv&&$url&/span& &span class=&o&&&&&/span& draw&span class=&o&&)&/span& &span class=&p&&&&/span&
&span class=&nb&&echo&/span& &span class=&nv&&$cursor&/span& & &span class=&nv&&$PLAYING_FILE&/span&
&span class=&k&&else&/span&
mplayerpause
&span class=&k&&fi&/span&
&span class=&o&&}&/span&
&span class=&k&&function&/span& initialize &span class=&o&&{&/span&
&span class=&c1&&#初始化函数&/span&
&span class=&k&&if&/span& &span class=&o&&[&/span& -p &span class=&nv&&$CONTROL_FILE&/span& &span class=&o&&]&/span& &span class=&p&&;&/span&&span class=&k&&then&/span&
&span class=&c1&&#如果存在mplayer控制管道，则删掉先&/span&
rm &span class=&nv&&$CONTROL_FILE&/span&
&span class=&k&&fi&/span&
&span class=&c1&&#创建mplayer控制管道&/span&
mkfifo &span class=&nv&&$CONTROL_FILE&/span&
&span class=&c1&&#初始化“正在播放”变量为-1&/span&
&span class=&nb&&echo&/span& -1 & &span class=&nv&&$PLAYING_FILE&/span&
&span class=&o&&}&/span&
&span class=&k&&function&/span& quitluoo &span class=&o&&{&/span&
mplayerquit
sleep &span class=&m&&1&/span& &span class=&c1&&#给予mplayer进程退出一点时间&/span&
tput rmcup
&span class=&nb&&exit&/span&
&span class=&o&&}&/span&
&span class=&c1&&#main&/span&
tput smcup
initialize
&span class=&nb&&echo&/span& -e &span class=&nv&&$NORMAL&/span&
&span class=&nb&&trap&/span& &span class=&s1&&'quitluoo'&/span& INT
&span class=&c1&&#捕获ctrl+c信号，调用quitluoo函数&/span&
loadsongs &span class=&nv&&$1&/span&
&span class=&k&&while&/span& true&span class=&p&&;&/span& &span class=&k&&do&/span&
&span class=&k&&done&/span&
&/code&&/pre&&/div&&p&接下来，好好享受音乐吧！&/p&&figure&&img src=&https://pic1.zhimg.com/v2-0fe56c37cdb216fac2e3a_b.jpg& data-rawwidth=&1864& data-rawheight=&1025& class=&origin_image zh-lightbox-thumb& width=&1864& data-original=&https://pic1.zhimg.com/v2-0fe56c37cdb216fac2e3a_r.jpg&&&/figure&&br&&p&是不是很强势？&/p&&p&本期代码已经更新到我的github上去了：&a href=&https://link.zhihu.com/?target=https%3A//github.com/nikoloss/linuxstuff/tree/master/luooplayer& class=& wrap external& target=&_blank& rel=&nofollow noreferrer&&nikoloss/linuxstuff&/a&&/p&
既然播放控制函数和界面渲染我们都弄出来了，那么能把两者串起来的就是交互功能模块了！一旦交互函数弄好了，整个播放器也就大功告成了。这一期就一气呵成，记录下落网播放器最后的收尾工作！在急于写代码之前，我们先明确一下交互功能，我的设计如下，w/s…
&figure&&img src=&https://pic4.zhimg.com/v2-71b18ba5ac414e0bbf3d_b.jpg& data-rawwidth=&595& data-rawheight=&328& class=&origin_image zh-lightbox-thumb& width=&595& data-original=&https://pic4.zhimg.com/v2-71b18ba5ac414e0bbf3d_r.jpg&&&/figure&&p&Linux Shell是一种基本功，由于怪异的语法加之较差的可读性，通常被Python等脚本代替。既然是基本功，那就需要掌握，毕竟学习Shell脚本的过程中，还是能了解到很多Linux系统的内容。&/p&&p&Linux脚本大师不是人人都可以达到的，但是用一些简单的Shell实现一些常见的基本功能还是很有必要的。&/p&&p&下面我介绍Linux下使用Shell处理文本时最常用的工具：&br&find、grep、xargs、sort、uniq、tr、cut、paste、wc、sed、awk；&br&提供的例子和参数都是最常用和最为实用的；&br&我对shell脚本使用的原则是命令单行书写，尽量不要超过2行；&br&如果有更为复杂的任务需求，还是考虑python吧；&/p&&h2&&b&1、find 文件查找&/b&&/h2&&p&查找txt和pdf文件&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . \( -name &*.txt& -o -name &*.pdf& \) -print
&/code&&/pre&&/div&&p&正则方式查找.txt和pdf&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . -regex
&.*\(\.txt|\.pdf\)$&
&/code&&/pre&&/div&&p&-iregex： 忽略大小写的正则&/p&&p&否定参数&br&查找所有非txt文本&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . ! -name &*.txt& -print
&/code&&/pre&&/div&&p&指定搜索深度&br&打印出当前目录的文件（深度为1）&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . -maxdepth 1 -type f
&/code&&/pre&&/div&&h2&&b&定制搜索&/b&&/h2&&p&按类型搜索：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . -type d -print
//只列出所有目录
&/code&&/pre&&/div&&p&-type f 文件 / l 符号链接&/p&&p&按时间搜索：&br&-atime 访问时间 (单位是天，分钟单位则是-amin，以下类似）&br&-mtime 修改时间 （内容被修改）&br&-ctime 变化时间 （元数据或权限变化）&br&最近7天被访问过的所有文件：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . -atime 7 -type f -print
&/code&&/pre&&/div&&p&按大小搜索：&br&w字 k M G&br&寻找大于2k的文件&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . -type f -size +2k
&/code&&/pre&&/div&&p&按权限查找：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . -type f -perm 644 -print //找具有可执行权限的所有文件
&/code&&/pre&&/div&&p&按用户查找：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . -type f -user weber -print// 找用户weber所拥有的文件
&/code&&/pre&&/div&&h2&&b&找到后的后续动作&/b&&/h2&&p&删除：&br&删除当前目录下所有的swp文件：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . -type f -name &*.swp& -delete
&/code&&/pre&&/div&&p&执行动作（强大的exec）&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . -type f -user root -exec chown weber {} \; //将当前目录下的所有权变更为weber
&/code&&/pre&&/div&&p&注：{}是一个特殊的字符串，对于每一个匹配的文件，{}会被替换成相应的文件名；&br&eg：将找到的文件全都copy到另一个目录：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find . -type f -mtime +10 -name &*.txt& -exec cp {} OLD \;
&/code&&/pre&&/div&&p&结合多个命令&br&tips: 如果需要后续执行多个命令，可以将多个命令写成一个脚本。然后 -exec 调用时执行脚本即可；&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&-exec ./commands.sh {} \;
&/code&&/pre&&/div&&h2&&b&-print的定界符&/b&&/h2&&p&默认使用'\n'作为文件的定界符；&br&-print0 使用'\0'作为文件的定界符，这样就可以搜索包含空格的文件；&/p&&h2&&b&2、grep 文本搜索&/b&&/h2&&p&grep match_patten file // 默认访问匹配行&/p&&p&常用参数&br&-o 只输出匹配的文本行 &b&VS &/b&-v 只输出没有匹配的文本行&br&-c 统计文件中包含文本的次数&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&grep -c &text& filename
&/code&&/pre&&/div&&p&-n 打印匹配的行号&br&-i 搜索时忽略大小写&br&-l 只打印文件名&/p&&p&在多级目录中对文本递归搜索(程序员搜代码的最爱）：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&grep &class& . -R -n
&/code&&/pre&&/div&&p&匹配多个模式&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&grep -e &class& -e &vitural& file
&/code&&/pre&&/div&&p&grep输出以\0作为结尾符的文件名：（-z）&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&grep &test& file* -lZ| xargs -0 rm
&/code&&/pre&&/div&&h2&&b&3、xargs 命令行参数转换&/b&&/h2&&p&xargs 能够将输入数据转化为特定命令的命令行参数；这样，可以配合很多命令来组合使用。比如grep，比如find；&/p&&p&将多行输出转化为单行输出&br&cat file.txt| xargs&br&\n 是多行文本间的定界符&/p&&p&将单行转化为多行输出&br&cat single.txt | xargs -n 3&br&-n：指定每行显示的字段数&/p&&h2&&b&xargs参数说明&/b&&/h2&&p&-d 定义定界符 （默认为空格 多行的定界符为 \n）&br&-n 指定输出为多行&br&-I {} 指定替换字符串，这个字符串在xargs扩展时会被替换掉,用于待执行的命令需要多个参数时&br&eg：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&cat file.txt | xargs -I {} ./command.sh -p {} -1
&/code&&/pre&&/div&&p&-0：指定\0为输入定界符&br&eg：统计程序行数&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&find source_dir/ -type f -name &*.cpp& -print0 |xargs -0 wc -l
&/code&&/pre&&/div&&h2&&b&4、sort 排序&/b&&/h2&&p&字段说明：&br&-n 按数字进行排序 VS -d 按字典序进行排序&br&-r 逆序排序&br&-k N 指定按第N列排序&br&eg：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&sort -nrk 1 data.txt
sort -bd data // 忽略像空格之类的前导空白字符
&/code&&/pre&&/div&&h2&&b&5、uniq 消除重复行&/b&&/h2&&p&消除重复行&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&sort unsort.txt | uniq
&/code&&/pre&&/div&&p&统计各行在文件中出现的次数&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&sort unsort.txt | uniq -c
&/code&&/pre&&/div&&p&找出重复行&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&sort unsort.txt | uniq -d
&/code&&/pre&&/div&&p&可指定每行中需要比较的重复内容：-s 开始位置 -w 比较字符数&/p&&h2&&b&6、用tr进行转换&/b&&/h2&&p&通用用法&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&echo 12345 | tr '0-9' '' //加解密转换，替换对应字符
cat text| tr '\t' ' '
//制表符转空格
&/code&&/pre&&/div&&p&tr删除字符&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&cat file | tr -d '0-9' // 删除所有数字
&/code&&/pre&&/div&&p&-c 求补集&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&cat file | tr -c '0-9' //获取文件中所有数字
cat file | tr -d -c '0-9 \n'
//删除非数字数据
&/code&&/pre&&/div&&p&tr压缩字符&br&tr -s 压缩文本中出现的重复字符；最常用于压缩多余的空格&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&cat file | tr -s ' '
&/code&&/pre&&/div&&p&字符类&br&tr中可用各种字符类：&br&alnum：字母和数字&br&alpha：字母&br&digit：数字&br&space：空白字符&br&lower：小写&br&upper：大写&br&cntrl：控制（非可打印）字符&br&print：可打印字符&br&使用方法：tr [:class:] [:class:]&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&eg: tr '[:lower:]' '[:upper:]'
&/code&&/pre&&/div&&h2&&b&7、cut 按列切分文本&/b&&/h2&&p&截取文件的第2列和第4列：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&cut -f2,4 filename
&/code&&/pre&&/div&&p&去文件除第3列的所有列：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&cut -f3 --complement filename
&/code&&/pre&&/div&&p&-d 指定定界符：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&cat -f2 -d&;& filename
&/code&&/pre&&/div&&p&cut 取的范围&br&N- 第N个字段到结尾&br&-M 第1个字段为M&br&N-M N到M个字段&/p&&p&cut 取的单位&br&-b 以字节为单位&br&-c 以字符为单位&br&-f 以字段为单位（使用定界符）&/p&&p&eg:&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&cut -c1-5 file //打印第一到5个字符
cut -c-2 file
//打印前2个字符
&/code&&/pre&&/div&&h2&&b&8、paste 按列拼接文本&/b&&/h2&&p&将两个文本按列拼接到一起;&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&cat file112cat file2
paste file1 file21 colin2 book
&/code&&/pre&&/div&&p&默认的定界符是制表符，可以用-d指明定界符&br&paste file1 file2 -d &,&&br&1,colin&br&2,book&/p&&h2&&b&9、wc 统计行和字符的工具&/b&&/h2&&p&wc -l file // 统计行数&br&wc -w file // 统计单词数&br&wc -c file // 统计字符数&/p&&h2&&b&10、sed 文本替换利器&/b&&/h2&&p&首处替换&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&seg 's/text/replace_text/' file
//替换每一行的第一处匹配的text
&/code&&/pre&&/div&&p&全局替换&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&seg 's/text/replace_text/g' file
&/code&&/pre&&/div&&p&默认替换后，输出替换后的内容，如果需要直接替换原文件,使用-i：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&seg -i 's/text/repalce_text/g' file
&/code&&/pre&&/div&&p&移除空白行：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&sed '/^$/d' file
&/code&&/pre&&/div&&p&变量转换&br&已匹配的字符串通过标记&来引用.&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&echo this is en example | seg 's/\w+/[&]/g'$&[this]
[is] [en] [example]
&/code&&/pre&&/div&&p&子串匹配标记&br&第一个匹配的括号内容使用标记 \1 来引用&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&sed 's/hello\([0-9]\)/\1/'
&/code&&/pre&&/div&&p&双引号求值&br&sed通常用单引号来引用；也可使用双引号，使用双引号后，双引号会对表达式求值：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&sed 's/$var/HLLOE/'
&/code&&/pre&&/div&&p&当使用双引号时，我们可以在sed样式和替换字符串中指定变量；&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&eg:p=patten
r=replaced
echo &line con a patten& | sed &s/$p/$r/g&$&line con a replaced
&/code&&/pre&&/div&&p&其它示例&br&字符串插入字符：将文本中每行内容（PEKSHA） 转换为 PEK/SHA&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&sed 's/^.\{3\}/&\//g' file
&/code&&/pre&&/div&&h2&&b&11、awk 数据流处理工具&/b&&/h2&&p&awk脚本结构&br&awk ' BEGIN{ statements } statements2 END{ statements } '&/p&&p&工作方式&br&1.执行begin中语句块；&br&2.从文件或stdin中读入一行，然后执行statements2，重复这个过程，直到文件全部被读取完毕；&br&3.执行end语句块；&/p&&h2&&b&print 打印当前行&/b&&/h2&&p&使用不带参数的print时，会打印当前行;&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&echo -e &line1\nline2& | awk 'BEGIN{print &start&} {print } END{ print &End& }'
&/code&&/pre&&/div&&p&print 以逗号分割时，参数以空格定界;&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&echo | awk ' {var1 = &v1& ; var2 = &V2&; var3=&v3&; \
print var1, var2 , var3; }'$&v1 V2 v3
&/code&&/pre&&/div&&p&使用-拼接符的方式（&&作为拼接符）;&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&echo | awk ' {var1 = &v1& ; var2 = &V2&; var3=&v3&; \
print var1&-&var2&-&var3; }'$&v1-V2-v3
&/code&&/pre&&/div&&h2&&b&特殊变量： NR NF $0 $1 $2&/b&&/h2&&p&NR:表示记录数量，在执行过程中对应当前行号；&br&NF:表示字段数量，在执行过程总对应当前行的字段数；&br&$0:这个变量包含执行过程中当前行的文本内容；&br&$1:第一个字段的文本内容；&br&$2:第二个字段的文本内容；&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&echo -e &line1 f2 f3\n line2 \n line 3& | awk '{print NR&:&$0&-&$1&-&$2}'
&/code&&/pre&&/div&&p&打印每一行的第二和第三个字段：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&awk '{print $2, $3}' file
&/code&&/pre&&/div&&p&统计文件的行数：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&awk ' END {print NR}' file
&/code&&/pre&&/div&&p&累加每一行的第一个字段：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&echo -e &1\n 2\n 3\n 4\n& | awk 'BEGIN{num = 0 ;
print &begin&;} {sum += $1;} END {print &==&; print sum }'
&/code&&/pre&&/div&&h2&&b&传递外部变量&/b&&/h2&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&var=1000echo | awk '{print vara}' vara=$var #
输入来自stdinawk '{print vara}' vara=$var file # 输入来自文件
&/code&&/pre&&/div&&h2&&b&用样式对awk处理的行进行过滤&/b&&/h2&&p&awk 'NR & 5' #行号小于5&br&awk 'NR==1,NR==4 {print}' file #行号等于1和4的打印出来&br&awk '/linux/' #包含linux文本的行（可以用正则表达式来指定，超级强大）&br&awk '!/linux/' #不包含linux文本的行&/p&&h2&&b&设置定界符&/b&&/h2&&p&使用-F来设置定界符（默认为空格）&br&awk -F: '{print $NF}' /etc/passwd&/p&&h2&&b&读取命令输出&/b&&/h2&&p&使用getline，将外部shell命令的输出读入到变量cmdout中；&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&echo | awk '{&grep root /etc/passwd& | print cmdout }'
&/code&&/pre&&/div&&h2&&b&在awk中使用循环&/b&&/h2&&p&for(i=0;i&10;i++){print $i;}&br&for(i in array){print array[i];}&/p&&p&eg:&br&以逆序的形式打印行：(tac命令的实现）&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&seq 9| \
awk '{lifo[NR] = $0; lno=NR} \
END{ for(;lno&-1;lno--){print lifo[lno];}
&/code&&/pre&&/div&&h2&&b&awk实现head、tail命令&/b&&/h2&&p&head:&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&awk 'NR&=10{print}' filename
&/code&&/pre&&/div&&p&tail:&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&awk '{buffer[NR%10] = $0;} END{for(i=0;i&11;i++){ \
print buffer[i %10]} } ' filename
&/code&&/pre&&/div&&h2&&b&打印指定列&/b&&/h2&&p&awk方式实现：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&ls -lrt | awk '{print $6}'
&/code&&/pre&&/div&&p&cut方式实现&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&ls -lrt | cut -f6
&/code&&/pre&&/div&&h2&&b&打印指定文本区域&/b&&/h2&&p&确定行号&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&seq 100| awk 'NR==4,NR==6{print}'
&/code&&/pre&&/div&&p&确定文本&br&打印处于start_pattern 和end_pattern之间的文本；&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&awk '/start_pattern/, /end_pattern/' filename
&/code&&/pre&&/div&&p&eg:&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&seq 100 | awk '/13/,/15/'cat /etc/passwd| awk '/mai.*mail/,/news.*news/'
&/code&&/pre&&/div&&h2&&b&awk常用内建函数&/b&&/h2&&p&index(string,search_string):返回search_string在string中出现的位置&br&sub(regex,replacement_str,string):将正则匹配到的第一处内容替换为replacement_&br&match(regex,string):检查正则表达式是否能够匹配字符串；&br&length(string)：返回字符串长度&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&echo | awk '{&grep root /etc/passwd& | print length(cmdout) }'
&/code&&/pre&&/div&&p&printf 类似c语言中的printf，对输出进行格式化&br&eg：&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&seq 10 | awk '{printf &-&%4s\n&, $1}'
&/code&&/pre&&/div&&h2&&b&12、迭代文件中的行、单词和字符&/b&&/h2&&h2&&b&1. 迭代文件中的每一行&/b&&/h2&&p&while 循环法&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&doecho $done & file.txt改成子shell:cat file.txt | (do echo $done)
&/code&&/pre&&/div&&p&awk法：&br&cat file.txt| awk '{print}'&/p&&h2&&b&2.迭代一行中的每一个单词&/b&&/h2&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&for word in $do echo $done
&/code&&/pre&&/div&&h2&&b&3. 迭代每一个字符&/b&&/h2&&p&${string:start_pos:num_of_chars}：从字符串中提取一个字符；(bash文本切片）&br&${#word}:返回变量word的长度&/p&&div class=&highlight&&&pre&&code class=&language-text&&&span&&/span&for((i=0;i&${#word};i++))doecho ${word:i:1);done
&/code&&/pre&&/div&&p&&br&&/p&&p&&b&来自：大CC&/b&&/p&&p&&b&链接：&a href=&https://link.zhihu.com/?target=http%3A//www.cnblogs.com/me115/p/3427319.html& class=& external& target=&_blank& rel=&nofollow noreferrer&&&span class=&invisible&&http://www.&/span&&span class=&visible&&cnblogs.com/me115/p/342&/span&&span class=&invisible&&7319.html&/span&&span class=&ellipsis&&&/span&&/a&&/b&&/p&&p&看一看，这么多能力你还差什么？&/p&&hr&&p&你想更深入了解学习Linux知识体系，你可以看一下我们花费了一个多月整理了上百小时的几百个知识点体系内容：&/p&&p&&a href=&https://link.zhihu.com/?target=http%3A//www.magedu.com/74163.html/%3Flinux_wenzhang_zhihu_xiujiang_linuxgaoxinxunlianying_& class=& wrap external& target=&_blank& rel=&nofollow noreferrer&&【超全整理】《Linux云计算从入门到精通》系列实战笔记全放送&/a&&/p&&p&&b&为了方便大家交流学习，我们特地组建了Linux运维学习的交流群，方便大家交流学习，目前已经有300个小伙伴在里面学习了，QQ群号码：&/b&&/p&
Linux Shell是一种基本功，由于怪异的语法加之较差的可读性，通常被Python等脚本代替。既然是基本功，那就需要掌握，毕竟学习Shell脚本的过程中，还是能了解到很多Linux系统的内容。Linux脚本大师不是人人都可以达到的，但是用一些简单的Shell实现一些常见…
&figure&&img src=&https://pic4.zhimg.com/50/v2-475b2e0b41b8cf84db2205_b.jpg& data-caption=&& data-size=&normal& data-rawwidth=&703& data-rawheight=&258& class=&origin_image zh-lightbox-thumb& width=&703& data-original=&https://pic4.zhimg.com/50/v2-475b2e0b41b8cf84db2205_r.jpg&&&/figure&&p&Python 库实在是太多了，已经多到不管是谁都不可能知道全部的 Python 库。这里面有很多库大家一定耳熟能详，比如 Pandas，Scikit-learn 和 Numpy，但是也有些库大家可能错过了，虽然它们有些老旧，但是同样非常有用。&/p&&p&&br&&/p&&p&我们今天就分享一些比较小众的 Python 库，建议 Python 老手也可以看一看，里面可能有那么一两个库你也没有见过。（库都送到嘴边了请你一定要尝尝&/p&&p&&br&&/p&&figure&&img src=&https://pic1.zhimg.com/50/v2-114eed85aedd0f4c71ca5_b.jpg& data-caption=&& data-size=&normal& data-rawwidth=&400& data-rawheight=&225& class=&content_image& width=&400&&&/figure&&p&&br&&/p&&h2&&b&Delorean&/b&&/h2&&p&Delorean 是一个很酷炫的日期时间库，是 Python 中用着最顺畅的日期时间库之一。它有点像 JavaScript 里的 moment，可以让你更简洁省心的解决 Python 中使用时间模块中遇到的问题，比如转换、操控和生成时间日期。&/p&&p&Delorean 这个名字也有点意思，源自科幻电影《回到未来》中那辆叫 Delorean 的梦幻鸥翼门汽车。&/p&&p&&br&&/p&&p&地址：&/p&&p&&a href=&//link.zhihu.com/?target=https%3A//delorean.readthedocs.io/en/latest/& class=& external& target=&_blank& rel=&nofollow noreferrer&&&span class=&invisible&&https://&/span&&span class=&visible&&delorean.readthedocs.io&/span&&span class=&invisible&&/en/latest/&/span&&span class=&ellipsis&&&/span&&/a&&/p&&p&&br&&/p&&div class=&highlight&&&pre&&code class=&language-python3&&&span class=&kn&&from&/span& &span class=&nn&&delorean&/span& &span class=&k&&import&/span& &span class=&n&&Delorean&/span&
&span class=&n&&EST&/span& &span class=&o&&=&/span& &span class=&s&&&US/Eastern&&/span&
&span class=&n&&d&/span& &span class=&o&&=&/span& &span class=&n&&Delorean&/span&&span class=&p&&(&/span&&span class=&n&&timezone&/span&&span class=&o&&=&/span&&span class=&n&&EST&/span&&span class=&p&&)&/span&
&/code&&/pre&&/div&&p&&br&&/p&&p&&br&&/p&&h2&&b&Prettytable&/b&&/h2&&p&很可能你没听过 Prettytable 这个库，因为它在 GoogleCode上···&/p&&p&简直就是编程圈里的西伯利亚，又偏又冷。&/p&&figure&&img src=&https://pic4.zhimg.com/50/v2-d9e2ffbec10aed395b767fb6b5de954d_b.jpg& data-caption=&& data-size=&normal& data-rawwidth=&252& data-rawheight=&200& class=&content_image& width=&252&&&/figure&&p&不过虽然身在犄角旮旯，Prettytable 却能帮我们在终端或浏览器上构造出很漂亮的外观。所以如果你在为 IPython Notebook 忙着找新插件，试试 Prettytable 用到你的 HTML_repr_上。&/p&&p&&br&&/p&&p&地址：&/p&&p&&a href=&//link.zhihu.com/?target=https%3A//code.google.com/archive/p/prettytable/& class=& external& target=&_blank& rel=&nofollow noreferrer&&&span class=&invisible&&https://&/span&&span class=&visible&&code.google.com/archive&/span&&span class=&invisible&&/p/prettytable/&/span&&span class=&ellipsis&&&/span&&/a&&/p&&p&&br&&/p&&h2&&b&Wget&/b&&/h2&&p&是不是有时候因为工作需要，自己要闷头写网络爬虫？有个叫 Wget 的库就是个很好的爬虫工具。想递归下载一个网站？爬取页面的所有图片？想避开 cookie 追踪？没问题，全都可以。&/p&&p&&br&&/p&&p&甚至在电影《社交网络》里连马克·扎克伯格都说：&/p&&p&首先是 Kirkland，它们开放了所有的东西，允许在他们的 Apache 配置上索引内容。所以用点 Wget 的小技巧就足够下载整个 Kirkland Facebook。小菜一碟！&/p&&p&&br&&/p&&p&Wdget 的 Python 版本非常简单易用：&/p&&p&&br&&/p&&div class=&highlight&&&pre&&code class=&language-python3&&&span class=&kn&&import&/span& &span class=&nn&&wget&/span&
&span class=&n&&wget&/span&&span class=&o&&.&/span&&span class=&n&&download&/span&&span class=&p&&(&/span&&span class=&s&&&http://www.cnn.com/&&/span&&span class=&p&&)&/span&
&span class=&c&&# 100% [............................................................................] 280385 / 280385&/span&
&/code&&/pre&&/div&&p&&br&&/p&&p&地址：&/p&&p&&a href=&//link.zhihu.com/?target=https%3A//pypi.org/project/wget/& class=& external& target=&_blank& rel=&nofollow noreferrer&&&span class=&invisible&&https://&/span&&span class=&visible&&pypi.org/project/wget/&/span&&span class=&invisible&&&/span&&/a&&/p&&p&&br&&/p&&h2&&b&PyMC&/b&&/h2&&p&不确定现在有多少人用 PyMC，现在 Scikit-learn 好像成了人见人爱的小天使（也确实很厉害），但是 PyMC 也值得我们的关爱。&/p&&p&&br&&/p&&p&PyMC 是一个用于贝叶斯分析的库，在很多数据科学和 Python 博客里以及卡梅隆·戴维森-皮隆的《Bayesian Methods for Hackers》里多次出现。&/p&&div class=&highlight&&&pre&&code class=&language-python3&&&span class=&kn&&from&/span& &span class=&nn&&pymc.examples&/span& &span class=&k&&import&/span& &span class=&n&&disaster_model&/span&
&span class=&kn&&from&/span& &span class=&nn&&pymc&/span& &span class=&k&&import&/span& &span class=&n&&MCMC&/span&
&span class=&n&&M&/span& &span class=&o&&=&/span& &span class=&n&&MCMC&/span&&span class=&p&&(&/span&&span class=&n&&disaster_model&/span&&span class=&p&&)&/span&
&span class=&n&&M&/span&&span class=&o&&.&/span&&span class=&n&&sample&/span&&span class=&p&&(&/span&&span class=&nb&&iter&/span&&span class=&o&&=&/span&&span class=&mi&&10000&/span&&span class=&p&&,&/span& &span class=&n&&burn&/span&&span class=&o&&=&/span&&span class=&mi&&1000&/span&&span class=&p&&,&/span& &span class=&n&&thin&/span&&span class=&o&&=&/span&&span class=&mi&&10&/span&&span class=&p&&)&/span&
&span class=&p&&[&/span&&span class=&o&&-----------------&/span&&span class=&mi&&100&/span&&span class=&o&&%-----------------&/span&&span class=&p&&]&/span& &span class=&mi&&10000&/span& &span class=&n&&of&/span& &span class=&mi&&10000&/span& &span class=&n&&complete&/span& &span class=&ow&&in&/span& &span class=&mf&&1.4&/span& &span class=&n&&sec&/span&
&/code&&/pre&&/div&&p&&br&&/p&&p&地址：&/p&&p&&a href=&//link.zhihu.com/?target=https%3A//pymc-devs.github.io/pymc/& class=& external& target=&_blank& rel=&nofollow noreferrer&&&span class=&invisible&&https://&/span&&span class=&visible&&pymc-devs.github.io/pym&/span&&span class=&invisible&&c/&/span&&span class=&ellipsis&&&/span&&/a&&/p&&p&&br&&/p&&h2&&b&Sh&/b&&/h2&&p&不能没让你看到 Sh 这个库你就走掉了。Sh 能让你将 shell 命令以函数导入 Python 中。如果有些工作在 bash 中很简单，但你不知道怎么在 Python 中完成（比如递归搜索文件），这时 Sh 就会超级有用。&/p&&div class=&highlight&&&pre&&code class=&language-python3&&&span class=&kn&&from&/span& &span class=&nn&&sh&/span& &span class=&k&&import&/span& &span class=&n&&find&/span&
&span class=&n&&find&/span&&span class=&p&&(&/span&&span class=&s&&&/tmp&&/span&&span class=&p&&)&/span&
&span class=&o&&/&/span&&span class=&n&&tmp&/span&&span class=&o&&/&/span&&span class=&n&&foo&/span&
&span class=&o&&/&/span&&span class=&n&&tmp&/span&&span class=&o&&/&/span&&span class=&n&&foo&/span&&span class=&o&&/&/span&&span class=&n&&file1&/span&&span class=&o&&.&/span&&span class=&n&&json&/span&
&span class=&o&&/&/span&&span class=&n&&tmp&/span&&span class=&o&&/&/span&&span class=&n&&foo&/span&&span class=&o&&/&/span&&span class=&n&&file2&/span&&span class=&o&&.&/span&&span class=&n&&json&/span&
&span class=&o&&/&/span&&span class=&n&&tmp&/span&&span class=&o&&/&/span&&span class=&n&&foo&/span&&span class=&o&&/&/span&&span class=&n&&file3&/span&&span class=&o&&.&/span&&span class=&n&&json&/span&
&span class=&o&&/&/span&&span class=&n&&tmp&/span&&span class=&o&&/&/span&&span class=&n&&foo&/span&&span class=&o&&/&/span&&span class=&n&&bar&/span&&span class=&o&&/&/span&&span class=&n&&file3&/span&&span class=&o&&.&/span&&span class=&n&&json&/span&
&/code&&/pre&&/div&&p&&br&&/p&&p&地址：&a href=&//link.zhihu.com/?target=https%3A//amoffat.github.io/sh/& class=& external& target=&_blank& rel=&nofollow noreferrer&&&span class=&invisible&&https://&/span&&span class=&visible&&amoffat.github.io/sh/&/span&&span class=&invisible&&&/span&&/a&&/p&&p&&br&&/p&&h2&&b&Fuzzywuzzy&/b&&/h2&&p&Fuzzywuzzy 是一款非常简单的 Python 库，花个两三分钟就能通读它的原理：&/p&&p&&a href=&//link.zhihu.com/?target=https%3A//github.com/seatgeek/fuzzywuzzy/blob/master/fuzzywuzzy/fuzz.py& class=& external& target=&_blank& rel=&nofollow noreferrer&&&span class=&invisible&&https://&/span&&span class=&visible&&github.com/seatgeek/fuz&/span&&span class=&invisible&&zywuzzy/blob/master/fuzzywuzzy/fuzz.py&/span&&span class=&ellipsis&&&/span&&/a& &/p&&p&&br&&/p&&p&Fuzzywuzzy 是一款很强大的模糊字符串匹配工具，很适合用于在不同数据库中创建特征向量或匹配记录。&/p&&p&&br&&/p&&p&地址：&/p&&p&&a href=&//link.zhihu.com/?target=http%3A//chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/& class=& external& target=&_blank& rel=&nofollow noreferrer&&&span class=&invisible&&http://&/span&&span class=&visible&&chairnerd.seatgeek.com/&/span&&span class=&invisible&&fuzzywuzzy-fuzzy-string-matching-in-python/&/span&&span class=&ellipsis&&&/span&&/a&&/p&&p&&br&&/p&&h2&&b&Progressbar&/b&&/h2&&p&Progressbar 是 Python 中的一个文本进度条程序库，用于展示长时间运行操作的过程，从视觉上提示你程序的处理进度。&/p&&p&&br&&/p&&p&Progressbar 非常简单易用，也很强大。如果系统支持，它还能自动启用一些功能，比如自动调整。&/p&&p&&br&&/p&&div class=&highlight&&&pre&&code class=&language-python3&&&span class=&kn&&from&/span& &span class=&nn&&progressbar&/span& &span class=&k&&import&/span& &span class=&n&&ProgressBar&/span&
&span class=&kn&&import&/span& &span class=&nn&&time&/span&
&span class=&n&&pbar&/span& &span class=&o&&=&/span& &span class=&n&&ProgressBar&/span&&span class=&p&&(&/span&&span class=&n&&maxval&/span&&span class=&o&&=&/span&&span class=&mi&&10&/span&&span class=&p&&)&/span&
&span class=&k&&for&/span& &span class=&n&&i&/span& &span class=&ow&&in&/span& &span class=&nb&&range&/span&&span class=&p&&(&/span&&span class=&mi&&1&/span&&span class=&p&&,&/span& &span class=&mi&&11&/span&&span class=&p&&):&/span&
&span class=&n&&pbar&/span&&span class=&o&&.&/span&&span class=&n&&update&/span&&span class=&p&&(&/span&&span class=&n&&i&/span&&span class=&p&&)&/span&
&span class=&n&&time&/span&&span class=&o&&.&/span&&span class=&n&&sleep&/span&&span class=&p&&(&/span&&span class=&mi&&1&/span&&span class=&p&&)&/span&
&span class=&n&&pbar&/span&&span class=&o&&.&/span&&span class=&n&&finish&/span&&span class=&p&&()&/span&
&span class=&c&&# 60% |######################################################## |&/span&
&/code&&/pre&&/div&&p&&br&&/p&&p&地址：&/p&&p&&a href=&//link.zhihu.com/?target=https%3A//github.com/niltonvolpato/python-progressbar& class=& external& target=&_blank& rel=&nofollow noreferrer&&&span class=&invisible&&https://&/span&&span class=&visible&&github.com/niltonvolpat&/span&&span class=&invisible&&o/python-progressbar&/span&&span class=&ellipsis&&&/span&&/a&&/p&&p&&br&&/p&&h2&&b&Colorama&/b&&/h2&&p&前面说到可以让程序有很棒的进度条，干嘛不再酷炫点，比如变成彩色！这点也很实用，可以在运行出错时提醒你。借助 Colorama 就能做到，使用起来很简单。将它放入脚本中，往颜色中添加任何你想打印的文本即可：&/p&&p&&br&&/p&&p&地址：&/p&&p&&a href=&//link.zhihu.com/?target=https%3A//pypi.org/project/colorama/& class=& external& target=&_blank& rel=&nofollow noreferrer&&&span class=&invisible&&https://&/span&&span class=&visible&&pypi.org/project/colora&/span&&span class=&invisible&&ma/&/span&&span class=&ellipsis&&&/span&&/a&&/p&&p&&br&&/p&&h2&&b&Uuid&/b&&/h2&&p&&br&&/p&&p&如果你在编程过程中需要唯一 ID，那么这个叫 UUID 的 Python 库绝对是个好工具，它实现了 1,3,4 和 5 版本的 UUID 标准，用起来很方便。&/p&&p&&br&&/p&&div class=&highlight&&&pre&&code class=&language-python3&&&span class=&kn&&import&/span& &span class=&nn&&uuid&/span&
&span class=&nb&&print&/span& &span class=&n&&uuid&/span&&span class=&o&&.&/span&&span class=&n&&uuid4&/span&&span class=&p&&()&/span&
&span class=&c&&# e7bafa3d-274e-4b0a-b9cc-db61&/span&
&/code&&/pre&&/div&&p&&br&&/p&&p&地址：&/p&&p&&a href=&//link.zhihu.com/?target=https%3A//docs.python.org/2/library/uuid.html& class=& external& target=&_blank& rel=&nofollow noreferrer&&&span class=&invisible&&https://&/span&&span class=&visible&&docs.python.org/2/libra&/span&&span class=&invisible&&ry/uuid.html&/span&&span class=&ellipsis&&&/span&&/a&&/p&&p&&br&&/p&&h2&&b&Bashplotlib&/b&&/h2&&p&Bashplotlib 是一款}