Repository navigation
Expand file tree
/
Copy pathatom.xml
More file actions
451 lines (244 loc) · 278 KB
/
Copy pathatom.xml
File metadata and controls
451 lines (244 loc) · 278 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>Whyyy</title>
<link href="/atom.xml" rel="self"/>
<link href="https://oceanoverflow.github.io/"/>
<updated>2018-07-02T10:48:13.921Z</updated>
<id>https://oceanoverflow.github.io/</id>
<author>
<name>Yangyi, Yi</name>
</author>
<generator uri="http://hexo.io/">Hexo</generator>
<entry>
<title>BarberShop</title>
<link href="https://oceanoverflow.github.io/2018/07/01/BarberShop/"/>
<id>https://oceanoverflow.github.io/2018/07/01/BarberShop/</id>
<published>2018-07-01T09:34:34.000Z</published>
<updated>2018-07-02T10:48:13.921Z</updated>
<content type="html"><![CDATA[<p>理发师问题是操作系统中一个非常经典的同步问题，假设理发店里有一位理发师，一把理发椅和5把供等候理发的顾客坐的椅子。如果当前没有顾客，则理发师躺在理发椅上睡觉，如果有顾客前来，他必须唤醒理发师，如果理发师正在理发但又有顾客前来，如果有空位置可坐，顾客就坐下来等待，否则则离开。</p><p>下面我们使用 <code>golang</code> 中的 <code>for select</code> 机制结合 <code>goroutine</code> 和 <code>channel</code> 来解决这个问题。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">const</span> (</span><br><span class="line"> CUTTING_TIME = <span class="number">20</span></span><br><span class="line"> BARBERS_COUNT = <span class="number">1</span></span><br><span class="line"> HALL_SITS_AMOUNT = <span class="number">5</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> Barber <span class="keyword">struct</span> {</span><br><span class="line"> name <span class="keyword">int</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> Client <span class="keyword">struct</span> {</span><br><span class="line"> name <span class="keyword">int</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>clientProducer</code> 模拟顾客前来，每隔一段随机的时间，前来一位顾客，<code>cutHear</code> 模拟理发师理发过程，理发结束时，向 <code>finished</code> 通道发送一个 <code>barber</code> ，表示该理发师已经空闲。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">clientProducer</span><span class="params">(clients <span class="keyword">chan</span> *Client)</span></span> {</span><br><span class="line"> <span class="keyword">for</span> {</span><br><span class="line"> time.Sleep(time.Duration(rand.Intn(<span class="number">28</span>) + <span class="number">7</span>)</span><br><span class="line"> clients <- &Client{}</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">cutHear</span><span class="params">(barber *Barber, client *Client, finished <span class="keyword">chan</span> *Barber)</span></span> {</span><br><span class="line"> fmt.Printf(<span class="string">"%v cutting hear for %v\n"</span>, barber.name, client.name)</span><br><span class="line"> time.Sleep(CUTTING_TIME * time.Millisecond)</span><br><span class="line"> finished <- barber</span><br><span class="line">}</span><br></pre></td></tr></table></figure><ol><li>若一顾客进入理发店，理发师正在为别人理发，且等待室有空椅子，该顾客就找张椅子按顺序坐下；</li><li>若一顾客进入理发店，理发师正在为别人理发，且所有椅子都被占用了，该顾客马上离开。</li><li>若一顾客进入理发店，理发师在睡觉，则叫醒理发师为该顾客理发；</li><li>如果理发师工作完，发现还有在等待的顾客，则呼唤顾客前来理发。</li><li>若没有要理发的顾客，则理发师去睡觉；</li></ol><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">BarberShop</span><span class="params">(clients <-<span class="keyword">chan</span> *Client)</span></span> {</span><br><span class="line"> freeBarbers := []*Barber{}</span><br><span class="line"> waitingClient := []*Client{}</span><br><span class="line"> syncBarberChan := <span class="built_in">make</span>(<span class="keyword">chan</span> *Barber)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> i := <span class="number">0</span>; i < BARBERS_COUNT; i++ {</span><br><span class="line"> freeBarbers = <span class="built_in">append</span>(freeBarbers, &Barber{})</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> {</span><br><span class="line"> <span class="keyword">select</span> {</span><br><span class="line"> <span class="keyword">case</span> client := <-clients:</span><br><span class="line"> <span class="keyword">if</span> <span class="built_in">len</span>(freeBarbers) == <span class="number">0</span> {</span><br><span class="line"> <span class="keyword">if</span> <span class="built_in">len</span>(waitingClient) < HALL_SITS_AMOUNT {</span><br><span class="line"> waitingClient = <span class="built_in">append</span>(waitingClient, client)</span><br><span class="line"> fmt.Printf(<span class="string">"Client is waiting in hall (%v)\n"</span>, <span class="built_in">len</span>(waitingClient))</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> fmt.Println(<span class="string">"No free space for client"</span>)</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> barber := freeBarbers[<span class="number">0</span>]</span><br><span class="line"> freeBarbers := freeBarbers[<span class="number">1</span>:]</span><br><span class="line"> fmt.Println(<span class="string">"Client goes to barber"</span>)</span><br><span class="line"> <span class="keyword">go</span> cutHear(barber, client, syncBarberChan)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">case</span> barber := <-syncBarberChan:</span><br><span class="line"> <span class="keyword">if</span> <span class="built_in">len</span>(waitingClient) > <span class="number">0</span> {</span><br><span class="line"> client := waitingClient[<span class="number">0</span>]</span><br><span class="line"> waitingClient = waitingClient[<span class="number">1</span>:]</span><br><span class="line"> fmt.Printf(<span class="string">"Take client from room (%v)\n"</span>, <span class="built_in">len</span>(waitingClient))</span><br><span class="line"> <span class="keyword">go</span> cutHear(barber, client, syncBarberChan)</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> fmt.Println(<span class="string">"Barber idle"</span>)</span><br><span class="line"> freeBarbers = <span class="built_in">append</span>(freeBarbers, barbers)</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>我们在 <code>main</code> 函数中起两个 <code>goroutine</code> ，分别代表顾客进程和理发师进程，理发师在工作8小时后退出，<code>clients</code> 通道作为两个 <code>goroutine</code> 沟通的媒介，类似于生产者和消费者模式。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> clients := <span class="built_in">make</span>(<span class="keyword">chan</span> *Client)</span><br><span class="line"> <span class="keyword">go</span> clientProducer(clients)</span><br><span class="line"> <span class="keyword">go</span> BarberShop(clients)</span><br><span class="line"> time.Sleep(<span class="number">8</span> * time.Hour)</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>理发师问题是操作系统中一个非常经典的同步问题，假设理发店里有一位理发师，一把理发椅和5把供等候理发的顾客坐的椅子。如果当前没有顾客，则理发师躺在理发椅上睡觉，如果有顾客前来，他必须唤醒理发师，如果理发师正在理发但又有顾客前来，如果有空位置可坐，顾客就坐下来等待，否则则离开。
</summary>
</entry>
<entry>
<title>Graphviz</title>
<link href="https://oceanoverflow.github.io/2018/06/27/Graphviz/"/>
<id>https://oceanoverflow.github.io/2018/06/27/Graphviz/</id>
<published>2018-06-27T13:37:09.000Z</published>
<updated>2018-06-27T13:45:10.019Z</updated>
<content type="html"><![CDATA[<p>最近看了许多比较酷炫的数据结构的实现，例如 <code>Trie</code> ， <code>Skip List</code> ，<code>B-Tree</code> 等，看代码的时候经常会想，要是有什么工具可以帮助我们将内存中的结构体绘制出来就好了，搜了一下 <code>github</code> ，发现早已经有大佬帮我们实现过了。</p><p>下面以二叉搜索树为例，来看一下如何使用 <code>graphviz</code> 绘制内存中的结构体。二叉树的定义非常简单，每个节点都有一个左节点和右节点，用于指向自己的左子树和右子树。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> Node <span class="keyword">struct</span> {</span><br><span class="line"> val <span class="keyword">int</span></span><br><span class="line"> left *Node</span><br><span class="line"> right *Node</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(n *Node)</span> <span class="title">Insert</span><span class="params">(val <span class="keyword">int</span>)</span> <span class="title">error</span></span> {</span><br><span class="line"> <span class="keyword">if</span> n == <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> errors.New(<span class="string">"Cannot insert a value into a nil tree"</span>)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">switch</span> {</span><br><span class="line"> <span class="keyword">case</span> val == n.val:</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line"> <span class="keyword">case</span> val < n.val:</span><br><span class="line"> <span class="keyword">if</span> n.left == <span class="literal">nil</span> {</span><br><span class="line"> n.left = &Node{val: val}</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> n.left.Insert(val)</span><br><span class="line"> <span class="keyword">case</span> val > n.val:</span><br><span class="line"> <span class="keyword">if</span> n.right == <span class="literal">nil</span> {</span><br><span class="line"> n.right = &Node{val: val}</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> n.right.Insert(val)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> Tree <span class="keyword">struct</span> {</span><br><span class="line"> Root *Node</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(t *Tree)</span> <span class="title">Insert</span><span class="params">(val <span class="keyword">int</span>)</span> <span class="title">error</span></span> {</span><br><span class="line"> <span class="keyword">if</span> t.root == <span class="literal">nil</span> {</span><br><span class="line"> t.Root = &Node{val: val}</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> t.Root.Insert(value)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>在 <code>main</code> 函数中我们构造一个二叉树，并向该树中随意插入一些值，最后我们使用 <code>memviz.Map(os.Stdout, &tree)</code> 将该树对应的 <code>DOT</code> 形态输出到标准输出中。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> <span class="string">"github.com/bradleyjkemp/memviz"</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> tree := &Tree{}</span><br><span class="line"> values := []<span class="keyword">int</span>{<span class="number">4</span>, <span class="number">2</span>, <span class="number">6</span>, <span class="number">1</span>, <span class="number">3</span>, <span class="number">5</span>, <span class="number">7</span>}</span><br><span class="line"> <span class="keyword">for</span> i := <span class="number">0</span>; i < <span class="built_in">len</span>(values); i++ {</span><br><span class="line"> err := tree.Insert(values[i])</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> log.Fatal(<span class="string">"Error inserting value '"</span>, values[i], <span class="string">"': "</span>, err)</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> memviz.Map(os.Stdout, &tree)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>利用 <code>unix pipe</code> 将 <code>go run main.go</code> 得到的结果输入到 <code>dot</code> 命令中。</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ go run main.go | dot -Tpng -o output.png | open output.png</span><br></pre></td></tr></table></figure><p><img src="https://image.ibb.co/bPod9o/output.png" alt="graphviz"></p><p> <code>dot</code> 命令相当于 <code>graphviz</code> 的一个子命令，它专门用于绘制有向图，除了 <code>dot</code> 之外，还有 <code>neato</code> ，<code>twopi</code> ，<code>circo</code> ，<code>fdp</code> ，<code>sfdp</code> ，<code>patchword</code> 和<code>osage</code> ，它们的用法与 <code>dot</code> 完全一致，只是它们的用处稍稍不同，例如 <code>neato</code> 比较适合用来绘制无向图。</p><p>下面我们来看一下（ <code>memviz.Map(os.Stdout, &tree)</code> ）的实现，该函数的实现主要依靠 <code>golang</code> 中的反射机制（ <code>reflect</code> ），利用 <code>reflect</code> 可以在运行时审查元素的类型，然后根据其类型，构造符合 <code>dot</code> 的语法的模版就行了。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">Map</span><span class="params">(w io.Writer, is ...<span class="keyword">interface</span>{})</span></span> {</span><br><span class="line"> <span class="keyword">var</span> iVals []reflect.Value</span><br><span class="line"> <span class="keyword">for</span> _, i := <span class="keyword">range</span> is {</span><br><span class="line"> iVal := reflect.ValueOf(i)</span><br><span class="line"> <span class="keyword">if</span> !iVal.CanAddr() {</span><br><span class="line"> <span class="keyword">if</span> iVal.Kind() != reflect.Ptr && iVal.Kind() != reflect.Interface {</span><br><span class="line"> fmt.Fprint(w, <span class="string">"error: cannot map unaddressable value"</span>)</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"> }</span><br><span class="line"> iVal = iVal.Elem()</span><br><span class="line"> }</span><br><span class="line"> iVals = <span class="built_in">append</span>(iVals, iVal)</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> m := &mapper{</span><br><span class="line"> w,</span><br><span class="line"> <span class="keyword">map</span>[nodeKey]nodeID{nilKey: <span class="number">0</span>},</span><br><span class="line"> <span class="keyword">map</span>[nodeKey]<span class="keyword">string</span>{nilKey: <span class="string">"nil"</span>},</span><br><span class="line"> <span class="number">2</span>,</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> fmt.Fprintln(w, <span class="string">"digraph structs {"</span>)</span><br><span class="line"> fmt.Fprintln(w, <span class="string">" node [shape=Mrecord];"</span>)</span><br><span class="line"> <span class="keyword">for</span> _, iVal := <span class="keyword">range</span> iVals {</span><br><span class="line"> m.mapValue(iVal, <span class="number">0</span>, <span class="literal">false</span>)</span><br><span class="line"> }</span><br><span class="line"> fmt.Fprintln(w, <span class="string">"}"</span>)</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>最近看了许多比较酷炫的数据结构的实现，例如 <code>Trie</code> ， <code>Skip List</code> ，<code>B-Tree</code> 等，看代码的时候经常会想，要是有什么工具可以帮助我们将内存中的结构体绘制出来就好了，搜了一下 <cod
</summary>
</entry>
<entry>
<title>CGO</title>
<link href="https://oceanoverflow.github.io/2018/06/26/CGO/"/>
<id>https://oceanoverflow.github.io/2018/06/26/CGO/</id>
<published>2018-06-26T08:34:33.000Z</published>
<updated>2018-07-03T12:18:05.760Z</updated>
<content type="html"><![CDATA[<p>cgo is an amazing technology which allows Go programs to interoperate with C libraries. Given a Go source file written with some special features, cgo outputs Go and C files that can be combined into a single Go package.</p><p>C code and Go code live in two different universes, cgo traverses the boundary between them. The transition is not free and depending on where it exists in your code, the cost could be inconsequential, or substantial. </p><p>There are some cases where cgo is unavoidable, most notably where you have to interoperate with a graphics driver or windowing system what is only available as a binary blob.</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> rand</span><br><span class="line"></span><br><span class="line"><span class="comment">// #include <stdlib.h></span></span><br><span class="line"><span class="keyword">import</span> <span class="string">"C"</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">Random</span><span class="params">()</span> <span class="title">int</span></span> {</span><br><span class="line"> <span class="keyword">var</span> r C.long = C.random()</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">int</span>(r)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">Seed</span><span class="params">(i <span class="keyword">int</span>)</span></span> {</span><br><span class="line"> C.srandom(C.<span class="keyword">uint</span>(i)) </span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="cgo-directives"><a href="#cgo-directives" class="headerlink" title="cgo directives"></a>cgo directives</h2><p>cgo is not a language nor a compiler. It’s a Foreign Function Interface (FFI), a mechanism we can use in Go to invoke functions and services written in a different language, specifically C.</p><p>The comment block right above the import “C” instruction is called a “preamble” and can contain actual C code, in this case an header inclusion. Once imported, the “C” pseudo-package lets us “jump” to the foreign code. You can build the example by invoking go build, the same as if it was plain Go.</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// #include <stdio.h></span></span><br><span class="line"><span class="comment">// #include <errno.h></span></span><br><span class="line"><span class="keyword">import</span> <span class="string">"C"</span></span><br></pre></td></tr></table></figure><p>CFLAGS, CPPFLAGS, CXXFLAGS, FFLAGS, LDFLAGS may be defined with pseudo <code>#cgo</code> directives within these comments to tweak the behavior of the C, C++ or Fortran compiler. Values defined in multiple directives are concatenated together. The directive can include a list of build constraints limiting its effect to systems satisfying one of the constraints.</p><p>cgo recognizes this comment. Any lines starting with <code>#cgo</code> followed by a space character are removed; these become directives for cgo. The remaining lines are used as a header when compiling the C parts of the package. In this case those lines are just a single <code>#include</code> statement, but they can be almost any C code. The <code>cgo</code> directives are used to provide flags for the compiler and linker when building the C parts of the package.</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// #cgo CFLAGS: -DPNG_DEBUG=1</span></span><br><span class="line"><span class="comment">// #cgo amd64 386 CFLAGS: -D86=1</span></span><br><span class="line"><span class="comment">// #cgo LDFLAGS: -lpng</span></span><br><span class="line"><span class="comment">// #include <png.h></span></span><br><span class="line"><span class="keyword">import</span> <span class="string">"C"</span></span><br></pre></td></tr></table></figure><p>Alternatively, CPPFLAGS and LDFLAGS may be obtained via the pkg-config tool using a ‘#cgo pkg-config:’ directive followed by the package names. For example:</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// #cgo pkg-config: png cairo</span></span><br><span class="line"><span class="comment">// #include <png.h></span></span><br><span class="line"><span class="keyword">import</span> <span class="string">"C"</span></span><br></pre></td></tr></table></figure><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// #cgo LDFLAGS -L${SRCDIR}/libs -l foo</span></span><br></pre></td></tr></table></figure><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// #cgo LDFLAGS -L/go/src/foo/libs -l foo</span></span><br></pre></td></tr></table></figure><p><code>-L</code> is the path to the directories containing the libraries. A search path for libraries.<br><code>-l</code> is the name of the library you want to link to.</p><p>For instance, if you want to link to the library <code>~/libs/libabc.a</code> you’d add:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">-L$(HOME)/libs -labc</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">| file | compiler |</span><br><span class="line">| ------------------------------- |:-----------------:|</span><br><span class="line">| xxx.c, xxx.s, xxx.S | C compiler |</span><br><span class="line">| xxx.cc, xxx.cpp, xxx.cxx | C++ compiler |</span><br><span class="line">| xxx.f, xxx.F, xxx.for, xxx.f90 | fortran compiler |</span><br><span class="line"></span><br><span class="line">Any .h, .hh, .hpp or .hxx files will not be compiled separately, but if these header files are changed, the C and C++ files will be recompiled.</span><br></pre></td></tr></table></figure><h2 id="tips-about-pkg-config"><a href="#tips-about-pkg-config" class="headerlink" title="tips about pkg-config"></a>tips about pkg-config</h2><p>pkg-config: return metainformation about installed libraries.</p><p>The <code>pkg-config</code> program is used to retrieve information about installed libraries in the system. It is typically used to compile and link against one or more libraries. Here is a typical usage scenario in a Makefile:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">program: program.c</span><br><span class="line"> cc program.c `pkg-config --cflags --libs gnomeui`</span><br></pre></td></tr></table></figure><p><code>pkg-config</code> retrieves information about packages from special metadata files. These files are named after the package, and has a <code>.pc</code> extension. On most systems, <code>pkg-config</code> looks in <code>/usr/lib/pkgconfig</code> , <code>/usr/share/pkgconfig</code> , <code>/usr/local/lib/pkgconfig</code> and <code>/usr/local/share/pkgconfig</code> for these files. It will additionally look in the colon-separated (on Windows, semi-colon-separated) list of directories specified by the <code>PKG_CONFIG_PATH</code> environment variable.</p><p>The package name specified on the <code>pkg-config</code> command line is defined to be the name of the metadata file, minus the <code>.pc</code> extension. If a library can install multiple versions simultaneously, it must give each version its own name (for example, GTK 1.2 might have the package name “gtk+” while GTK 2.0 has “gtk-2.0”).</p><p>In addition to specifying a package name on the command line, the full path to a given <code>.pc</code> file may be given instead. This allows a user to directly query a particular <code>.pc</code> file.</p><p><code>/usr/local/lib/pkgconfig/libuv.pc</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">prefix=/usr/local/Cellar/libuv/1.20.0</span><br><span class="line">exec_prefix=${prefix}</span><br><span class="line">libdir=${exec_prefix}/lib</span><br><span class="line">includedir=${prefix}/include</span><br><span class="line"></span><br><span class="line">Name: libuv</span><br><span class="line">Version: 1.20.0</span><br><span class="line">Description: multi-platform support library with a focus on asynchronous I/O.</span><br><span class="line">URL: http://libuv.org/</span><br><span class="line"></span><br><span class="line">Libs: -L${libdir} -luv -lphread -ldl</span><br><span class="line">Cflags: -I${includedir}</span><br></pre></td></tr></table></figure><h2 id="basic-data-type-conversion"><a href="#basic-data-type-conversion" class="headerlink" title="basic data type conversion"></a>basic data type conversion</h2><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line">| C | Go |</span><br><span class="line">| ------------------- |:---------------:|</span><br><span class="line">| <span class="keyword">char</span> | C.<span class="keyword">char</span> |</span><br><span class="line">| <span class="keyword">signed</span> <span class="keyword">char</span> | C.schar |</span><br><span class="line">| <span class="keyword">unsigned</span> <span class="keyword">char</span> | C.uchar |</span><br><span class="line">| <span class="keyword">short</span> | C.<span class="keyword">short</span> |</span><br><span class="line">| <span class="keyword">unsigned</span> <span class="keyword">short</span> | C.ushort |</span><br><span class="line">| <span class="keyword">int</span> | C.<span class="keyword">int</span> |</span><br><span class="line">| <span class="keyword">unsigned</span> <span class="keyword">int</span> | C.uint |</span><br><span class="line">| <span class="keyword">long</span> | C.<span class="keyword">long</span> |</span><br><span class="line">| <span class="keyword">unsigned</span> <span class="keyword">long</span> | C.ulong |</span><br><span class="line">| <span class="keyword">long</span> <span class="keyword">long</span> | C.longlong |</span><br><span class="line">| <span class="keyword">unsigned</span> <span class="keyword">long</span> <span class="keyword">long</span> | C.ulonglong |</span><br><span class="line">| <span class="keyword">float</span> | C.<span class="keyword">float</span> |</span><br><span class="line">| <span class="keyword">double</span> | C.<span class="keyword">double</span> |</span><br><span class="line">| <span class="keyword">complex</span> <span class="keyword">float</span> | C.complexfloat |</span><br><span class="line">| <span class="keyword">complex</span> <span class="keyword">double</span> | C.complexdouble |</span><br><span class="line">| <span class="keyword">long</span> <span class="keyword">long</span> | C.longlong |</span><br><span class="line">| <span class="keyword">void</span> * | unsafe.Pointer |</span><br><span class="line">| <span class="keyword">__int128_t</span> | [<span class="number">16</span>]byte |</span><br><span class="line">| <span class="keyword">__uint128_t</span> | [<span class="number">16</span>]byte |</span><br><span class="line">| few special C types | uintptr |</span><br></pre></td></tr></table></figure><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">| C | Go |</span><br><span class="line">| ---------------- |:-----------------:|</span><br><span class="line">| <span class="keyword">sizeof</span>(T) | C.sizeof_T |</span><br><span class="line">| <span class="keyword">sizeof</span>(struct T) | C.sizeof_struct_T |</span><br><span class="line">| <span class="class"><span class="keyword">struct</span> <span class="title">X</span> | <span class="title">C</span>.<span class="title">struct_X</span> |</span></span><br><span class="line"><span class="class">| <span class="title">union</span> <span class="title">X</span> | <span class="title">C</span>.<span class="title">union_X</span> |</span></span><br><span class="line"><span class="class">| <span class="title">enum</span> <span class="title">X</span> | <span class="title">C</span>.<span class="title">enum_X</span> |</span></span><br></pre></td></tr></table></figure><h2 id="string-conversion-between-C-and-Go"><a href="#string-conversion-between-C-and-Go" class="headerlink" title="string conversion between C and Go"></a>string conversion between C and Go</h2><p>Unlike Go, C doesn’t have an explicit string type. Strings in C are represented by a zero-terminated array of chars. Conversion between Go and C strings is done with the C.CString, C.GoString, and C.GoStringN functions. These conversions make a copy of the string data.</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// Go string to C string</span></span><br><span class="line"><span class="comment">// The C string is allocated in the C heap using malloc.</span></span><br><span class="line"><span class="comment">// It is the caller's responsibility to arrange for it to be</span></span><br><span class="line"><span class="comment">// freed, such as by calling C.free (be sure to include stdlib.h if C.free is needed).</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">C</span>.<span class="title">CString</span><span class="params">(<span class="keyword">string</span>)</span> *<span class="title">C</span>.<span class="title">char</span></span></span><br><span class="line"><span class="function"></span></span><br><span class="line"><span class="function">// <span class="title">Go</span> []<span class="title">byte</span> <span class="title">slice</span> <span class="title">to</span> <span class="title">C</span> <span class="title">array</span></span></span><br><span class="line"><span class="function">// <span class="title">The</span> <span class="title">C</span> <span class="title">array</span> <span class="title">is</span> <span class="title">allocated</span> <span class="title">in</span> <span class="title">the</span> <span class="title">C</span> <span class="title">heap</span> <span class="title">using</span> <span class="title">malloc</span>.</span></span><br><span class="line"><span class="function">// <span class="title">It</span> <span class="title">is</span> <span class="title">the</span> <span class="title">caller</span>'<span class="title">s</span> <span class="title">responsibility</span> <span class="title">to</span> <span class="title">arrange</span> <span class="title">for</span> <span class="title">it</span> <span class="title">to</span> <span class="title">be</span></span></span><br><span class="line"><span class="function">// <span class="title">freed</span>, <span class="title">such</span> <span class="title">as</span> <span class="title">by</span> <span class="title">calling</span> <span class="title">C</span>.<span class="title">free</span> <span class="params">(be sure to include stdlib.h <span class="keyword">if</span> C.free is needed)</span>.</span></span><br><span class="line"><span class="function"><span class="title">func</span> <span class="title">C</span>.<span class="title">CBytes</span><span class="params">([]<span class="keyword">byte</span>)</span> <span class="title">unsafe</span>.<span class="title">Pointer</span></span></span><br></pre></td></tr></table></figure><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// C string to Go string</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">C</span>.<span class="title">GoString</span><span class="params">(*C.char)</span> <span class="title">string</span></span></span><br><span class="line"><span class="function"></span></span><br><span class="line"><span class="function">// <span class="title">C</span> <span class="title">data</span> <span class="title">with</span> <span class="title">explicit</span> <span class="title">length</span> <span class="title">to</span> <span class="title">Go</span> <span class="title">string</span></span></span><br><span class="line"><span class="function"><span class="title">func</span> <span class="title">C</span>.<span class="title">GoStringN</span><span class="params">(*C.char, C.<span class="keyword">int</span>)</span> <span class="title">string</span></span></span><br><span class="line"><span class="function"></span></span><br><span class="line"><span class="function">// <span class="title">C</span> <span class="title">data</span> <span class="title">with</span> <span class="title">explicit</span> <span class="title">length</span> <span class="title">to</span> <span class="title">Go</span> []<span class="title">byte</span></span></span><br><span class="line"><span class="function"><span class="title">func</span> <span class="title">C</span>.<span class="title">GoBytes</span><span class="params">(unsafePinter, C.<span class="keyword">int</span>)</span> []<span class="title">byte</span></span></span><br></pre></td></tr></table></figure><h2 id="memory-management"><a href="#memory-management" class="headerlink" title="memory management"></a>memory management</h2><p>Memory allocations made by C code are not known to Go’s memory manager. When you create a C string with C.CString (or any C memory allocation) you must remember to free the memory when you’re done with it by calling C.free.</p><p>The call to C.CString returns a pointer to the start of the char array, so before the function exits we convert it to an unsafe.Pointer and release the memory allocation with C.free. A common idiom in cgo programs is to defer the free immediately after allocating (especially when the code that follows is more complex than a single function call), as in this rewrite of Print:</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">Print</span><span class="params">(s <span class="keyword">string</span>)</span></span> {</span><br><span class="line"> cs := C.CString(s)</span><br><span class="line"> <span class="keyword">defer</span> C.free(unsafe.Pointer(cs))</span><br><span class="line"> C.fputs(cs, (*C.FILE)(C.stdout))</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p> A standard way to do this follows.</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// #include <stdlib.h></span></span><br><span class="line"><span class="keyword">import</span> <span class="string">"C"</span></span><br><span class="line"><span class="keyword">import</span> <span class="string">"unsafe"</span></span><br><span class="line">...</span><br><span class="line"> <span class="keyword">var</span> cmsg *C.char = C.CString(<span class="string">"hi"</span>)</span><br><span class="line"> <span class="keyword">defer</span> C.free(unsafe.Pointer(cmsg))</span><br><span class="line"> <span class="comment">// do something with the C string</span></span><br></pre></td></tr></table></figure><h2 id="handle-C-return-error-in-Go"><a href="#handle-C-return-error-in-Go" class="headerlink" title="handle C return error in Go"></a>handle C return error in Go</h2><p>Any C function (even void functions) may be called in a multiple assigned context to retrieve both the return value (if any) and the C errno variable as an error (use _ to skip the result value if the function returns void). For example.</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment">The sqrt() function compute the non-negative square root of x</span></span><br><span class="line"><span class="comment">sqrt(-0) returns -0</span></span><br><span class="line"><span class="comment">sqrt(x) returns a NaN and generates a domain error for x < 0</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><math.h></span></span></span><br><span class="line"><span class="keyword">double</span></span><br><span class="line"><span class="built_in">sqrt</span>(<span class="keyword">double</span> x);</span><br></pre></td></tr></table></figure><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">n, err := C.sqrt(<span class="number">-1</span>)</span><br><span class="line">_, err = C.voidFunc()</span><br></pre></td></tr></table></figure><h2 id="call-Go-code-in-C-code"><a href="#call-Go-code-in-C-code" class="headerlink" title="call Go code in C code"></a>call Go code in C code</h2><p>Using //export in a file places a restriction on the preamble: since it is copied into two different C. Output files, it must not contain any definitions, only declarations.</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//export MyFunction</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">MyFunction</span><span class="params">(arg1, arg2 <span class="keyword">int</span>, arg3 <span class="keyword">string</span>)</span> <span class="title">int64</span></span> {...}</span><br><span class="line"></span><br><span class="line"><span class="comment">//export MyFunction2</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">MyFunction2</span><span class="params">(arg1, arg2 <span class="keyword">int</span>, arg3 <span class="keyword">string</span>)</span> <span class="params">(<span class="keyword">int64</span>, *C.char)</span></span> {...}</span><br></pre></td></tr></table></figure><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">extern</span> int64 <span class="title">MyFunction</span><span class="params">(<span class="keyword">int</span> arg1, <span class="keyword">int</span> arg2, GoString arg3)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">extern</span> struct MyFunction2_return <span class="title">MyFunction2</span><span class="params">(<span class="keyword">int</span> arg1, <span class="keyword">int</span> arg2, GoString arg3)</span></span>;</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>cgo is an amazing technology which allows Go programs to interoperate with C libraries. Given a Go source file written with some special
</summary>
</entry>
<entry>
<title>Trie</title>
<link href="https://oceanoverflow.github.io/2018/06/25/Trie/"/>
<id>https://oceanoverflow.github.io/2018/06/25/Trie/</id>
<published>2018-06-25T14:25:54.000Z</published>
<updated>2018-06-26T08:47:30.860Z</updated>
<content type="html"><![CDATA[<p>假设我们要实现一个类似于有道词典这样的字典，开始时使用哈希表来存储键值对，后来在实现前缀搜索，模糊匹配等相关功能时，发现使用哈希表作为底层存储的数据结构有一个致命的缺点，基于哈希表的前缀搜索，模糊匹配实现难度非常之大，所以决定换一种数据结构来存储数据。下面我们来介绍一下 <code>Trie</code> 树， <code>Trie</code> 树也叫字典树，它非常适合我们上面说的场景，作为一种树形结构，还经常被用于统计，排序和保存大量的字符串等场景。</p><p><img src="https://cdn-images-1.medium.com/max/1600/1*f89l46VKjKPgnecqOw4V_w.png" alt="trie"></p><h2 id="data-structure"><a href="#data-structure" class="headerlink" title="data structure"></a>data structure</h2><p><code>Trie</code> 是一棵R叉树，可以用于保存大量的字符串，每棵树都有一个根节点（ <code>root</code> ）。非叶节点存储一个 <code>rune</code> ，除了指向父节点的指针 <code>parent</code> 之外，并还包含一系列子节点 <code>children</code> ，使用 <code>map</code> 来存储。叶节点标志着一个字符串的结束，其 <code>term</code> 标记为 <code>true</code> ，叶节点中还可以存储一些元数据（ <code>meta</code> ），来表示该字符串的含义。从根节点开始一直到叶节点就可以形成一个完整的字符串。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> Node <span class="keyword">struct</span> {</span><br><span class="line"> val <span class="keyword">rune</span></span><br><span class="line"> term <span class="keyword">bool</span></span><br><span class="line"> depth <span class="keyword">int</span></span><br><span class="line"> meta <span class="keyword">interface</span>{}</span><br><span class="line"> parent *Node</span><br><span class="line"> children <span class="keyword">map</span>[<span class="keyword">rune</span>]*Node</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> Trie <span class="keyword">struct</span> {</span><br><span class="line"> root *Node</span><br><span class="line"> size <span class="keyword">int</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">New</span><span class="params">()</span> *<span class="title">Trie</span></span> {</span><br><span class="line"> <span class="keyword">return</span> &Trie{</span><br><span class="line"> root: &Node{depth: <span class="number">0</span>, children: <span class="built_in">make</span>(<span class="keyword">map</span>[<span class="keyword">rune</span>]*Node)}</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="helper-function"><a href="#helper-function" class="headerlink" title="helper function"></a>helper function</h2><p>为了方便后面的搜索操作，我们需要实现两个 <code>helper function</code> ，第一个就是计算一个字符串的掩码，例如对 <code>apple</code> 计算掩码时，分别对’a’，’p’，’p’，’l’，’e’ 这几个 <code>rune</code> 进行位运算，最后返回一个 <code>uint64</code> ，该掩码在搜索中起着非常重要的作用，因为每个节点都会存储一个掩码（ <code>mask</code> ），我们可以利用该掩码判断该节点及后续字节点存储了什么字符。<code>findNode</code> 方法帮助我们搜索 <code>runes</code> 字符数组中最后一个字符在树中对应的节点。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">maskruneslice</span><span class="params">(rs []<span class="keyword">rune</span>)</span> <span class="title">uint64</span></span> {</span><br><span class="line"> <span class="keyword">var</span> m <span class="keyword">uint64</span></span><br><span class="line"> <span class="keyword">for</span> _, r := <span class="keyword">range</span> rs {</span><br><span class="line"> m |= <span class="keyword">uint64</span>(<span class="number">1</span>) << <span class="keyword">uint64</span>(r-<span class="string">'a'</span>)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> m</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">findNode</span><span class="params">(node *Node, runes []<span class="keyword">rune</span>)</span> *<span class="title">Node</span></span> {</span><br><span class="line"> <span class="keyword">if</span> node == <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> <span class="built_in">len</span>(runes) == <span class="number">0</span> {</span><br><span class="line"> <span class="keyword">return</span> node</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> n, ok := node.Children()[runes[<span class="number">0</span>]]</span><br><span class="line"> <span class="keyword">if</span> !ok {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">var</span> nrunes []<span class="keyword">rune</span></span><br><span class="line"> <span class="keyword">if</span> <span class="built_in">len</span>(runes) > <span class="number">1</span> {</span><br><span class="line"> nrunes = runes[<span class="number">1</span>:]</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> nrunes = runes[<span class="number">0</span>:<span class="number">0</span>]</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> findNode(n, nrunes)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="Basic-Operation"><a href="#Basic-Operation" class="headerlink" title="Basic Operation"></a>Basic Operation</h2><p><code>Trie</code> 树的基本操作有三个，<code>Add</code> ，<code>Find</code> 和 <code>Del</code> 。</p><p><code>Add</code> 方法将 <code>key</code> 字符串加入到树中。对于每一个要加入树中的字符串中的 <code>rune</code> ， 我们构造一个节点，如果相应的 <code>rune</code> 已经存在，此时只需要更新其 <code>mask</code> 值，如果该 <code>rune</code> 不存在，则构建一个中间节点 <code>node = node.NewChild(r, bitmask, nil, false)</code> ，在所有中间节点都构建完毕之后，我们构建一个叶节点结束 <code>Add</code> 操作，叶节点的 <code>val</code> 值设为 <code>nul</code> ，<code>term</code> 为 <code>true</code> ，并存储相应的元数据 <code>meta</code> 。 </p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">const</span> nul = <span class="number">0x0</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(t *Trie)</span> <span class="title">Add</span><span class="params">(key <span class="keyword">string</span>, meta <span class="keyword">interface</span>)</span> *<span class="title">Node</span></span> {</span><br><span class="line"> t.size++</span><br><span class="line"> runes := []<span class="keyword">rune</span>(key)</span><br><span class="line"> bitmask := maskruneslice(runes)</span><br><span class="line"> node := t.root</span><br><span class="line"> node.mask |= bitmask</span><br><span class="line"> <span class="keyword">for</span> i := <span class="keyword">range</span> runes {</span><br><span class="line"> r := runes[i]</span><br><span class="line"> bitmask := maskruneslice(runes[i:])</span><br><span class="line"> <span class="keyword">if</span> n, ok := node.children[r]; ok {</span><br><span class="line"> node = n</span><br><span class="line"> node.mask |= bitmask</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> node = node.NewChild(r, bitmask, <span class="literal">nil</span>, <span class="literal">false</span>)</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> node = node.NewChild(nul, <span class="number">0</span>, meta, <span class="literal">true</span>)</span><br><span class="line"> <span class="keyword">return</span> node</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(n *Node)</span> <span class="title">NewChild</span><span class="params">(val <span class="keyword">rune</span>, bitmask <span class="keyword">uint64</span>, meta <span class="keyword">interface</span>{}, term <span class="keyword">bool</span>)</span> *<span class="title">Node</span></span> {</span><br><span class="line"> node := &Node{</span><br><span class="line"> val: val,</span><br><span class="line"> bitmask: bitmask,</span><br><span class="line"> term: term,</span><br><span class="line"> meta: meta,</span><br><span class="line"> parent: n,</span><br><span class="line"> children: <span class="built_in">make</span>(<span class="keyword">map</span>[<span class="keyword">rune</span>]*Node),</span><br><span class="line"> depth: n.depth + <span class="number">1</span>,</span><br><span class="line"> }</span><br><span class="line"> n.children[val] = node</span><br><span class="line"> n.mask |= bitmask</span><br><span class="line"> <span class="keyword">return</span> node</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>Find</code> 方法判断一个特定的字符串是否出现在该树中，我们先通过 <code>findNode</code> 找到相应的 <code>node</code> ，<code>Find</code> 方法用于查找一个完整的字符串，例如我们要查找 <code>app</code> ，但树中只有 <code>apple</code> 这个字符串，所以 <code>findNode</code> 调用时应该返回说不存在 <code>app</code> 这个字符串。当查找 <code>app</code> 这个字符串时，<code>findNode</code> 方法会返回最后一个 <code>p</code> 对应的节点，但这并不说明 <code>app</code> 存在于树中，我们还需要判断该节点是否为叶节点 <code>node.term == true</code> ，如果不是叶节点，则返回 <code>nil</code> 。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(t *Trie)</span> <span class="title">Find</span><span class="params">(key <span class="keyword">string</span>)</span> <span class="params">(*Node, <span class="keyword">bool</span>)</span></span> {</span><br><span class="line"> node := findNode(t.Root(), []<span class="keyword">rune</span>(key))</span><br><span class="line"> <span class="keyword">if</span> node == <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span>, <span class="literal">false</span></span><br><span class="line"> }</span><br><span class="line"> node, ok := node.Children()[nul]</span><br><span class="line"> <span class="keyword">if</span> !ok || !node.term {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span>, <span class="literal">false</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> node, <span class="literal">true</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>Remove</code> 方法同样通过 <code>findNode</code> 方法找到相应的节点，然后从该节点向根节点的方向依次删除相应的 <code>rune</code> ，主要目的是为了更新每个中间节点的<code>mask</code> 值。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(t *Trie)</span> <span class="title">Remove</span><span class="params">(key <span class="keyword">string</span>)</span></span> {</span><br><span class="line"> <span class="keyword">var</span> (</span><br><span class="line"> i <span class="keyword">int</span></span><br><span class="line"> rs = []<span class="keyword">rune</span>(key)</span><br><span class="line"> node = findNode(t.Root(), []<span class="keyword">rune</span>(key))</span><br><span class="line"> )</span><br><span class="line"></span><br><span class="line"> t.size--</span><br><span class="line"> <span class="keyword">for</span> n := node.Parent(); p != <span class="literal">nil</span>; n = n.Parent() {</span><br><span class="line"> i++</span><br><span class="line"> <span class="keyword">if</span> <span class="built_in">len</span>(n.Children()) > <span class="number">1</span> {</span><br><span class="line"> r := rs[<span class="built_in">len</span>(rs)-i]</span><br><span class="line"> n.RemoveChild(r)</span><br><span class="line"> <span class="keyword">break</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(n *Node)</span> <span class="title">RemoveChild</span><span class="params">(r <span class="keyword">rune</span>)</span></span> {</span><br><span class="line"> <span class="built_in">delete</span>(n.children, r)</span><br><span class="line"> <span class="keyword">for</span> nd := n.parent; nd != <span class="literal">nil</span>; nd = nd.parent {</span><br><span class="line"> nd.mask ^= nd.mask</span><br><span class="line"> nd.mask |= <span class="keyword">uint64</span>(<span class="number">1</span>) << <span class="keyword">uint64</span>(nd.val - <span class="string">'a'</span>)</span><br><span class="line"> <span class="keyword">for</span> _, c := <span class="keyword">range</span> nd.children {</span><br><span class="line"> nd.mask |= c.mask</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="Collect"><a href="#Collect" class="headerlink" title="Collect"></a>Collect</h2><p>有时候我们搜索单词的时候只记得该单词的前缀，例如我们要搜索 <code>append</code> 这个单词，但是我们只记得 <code>app</code> 这个前缀，我们可以使用 <code>collect</code> 方法帮我们收集所有 <code>app</code> 开头的单词，换句话说就是特定节点下面所有可能的叶节点的集合，开始收集时我们构造一个队列，该队列中仅有 <code>node</code> 一个节点，然后使用广度优先搜索的方式（层序遍历），首先将队列中的第一个节点出队，然后将该节点所有的子节点加入该队列中，如果节点为叶节点，则从该叶节点开始一直向上遍历到根节点的下一层节点（ <code>p.depth == 1</code> ），直到完成所有单词的收集为止。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">collect</span><span class="params">(node *Node)</span> []<span class="title">string</span></span> {</span><br><span class="line"> <span class="keyword">var</span> (</span><br><span class="line"> keys []<span class="keyword">string</span></span><br><span class="line"> n *Node</span><br><span class="line"> i <span class="keyword">int</span></span><br><span class="line"> )</span><br><span class="line"> nodes := []*Node{node}</span><br><span class="line"> <span class="keyword">for</span> l := <span class="built_in">len</span>(nodes); l != <span class="number">0</span>; l = <span class="built_in">len</span>(nodes) {</span><br><span class="line"> i := l - <span class="number">1</span></span><br><span class="line"> n = nodes[i]</span><br><span class="line"> nodes = nodes[:i]</span><br><span class="line"> <span class="keyword">for</span> _, c := <span class="keyword">range</span> n.children {</span><br><span class="line"> nodes = <span class="built_in">append</span>(nodes, c)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> n.term {</span><br><span class="line"> word := <span class="string">""</span></span><br><span class="line"> <span class="keyword">for</span> p := n.parent; p.depth != <span class="number">0</span>; p = p.parent {</span><br><span class="line"> word = <span class="keyword">string</span>(p.val) + word</span><br><span class="line"> }</span><br><span class="line"> keys = <span class="built_in">append</span>(keys, word)</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> keys</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="Fuzzy-Collect"><a href="#Fuzzy-Collect" class="headerlink" title="Fuzzy Collect"></a>Fuzzy Collect</h2><p>除了比较准确的前缀匹配之外，有时候我们可能还需要模糊匹配，再举 <code>append</code> 这个单词为例子，我们可能只记得这个单词包含了 <code>apd</code> 这样的模式，使用 <code>Trie</code> 树我们可以非常轻松地实现模糊匹配功能，<code>fuzzycollect</code> 同样利用层序遍历，在每一个循环开始时，对队列中的第一个节点进行出队操作，如果相应的掩码不包含在正在遍历的节点中 <code>(p.node.mask & m) != m</code> ，说明其后面的节点中不包含所有 <code>partial</code> 中包含的 <code>rune</code> 值，直接进入下一个循环 ，如果 <code>p.idx == len(partial)</code> ，说明我们找到了与 <code>partial</code> 相匹配的一个单词，此时进行 <code>collect</code> 操作 。最后同样需要将该节点的子节点加入队列中以进行下一轮的层序遍历。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> potentialSubtree <span class="keyword">struct</span> {</span><br><span class="line"> idx <span class="keyword">int</span></span><br><span class="line"> node *Node</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">fuzzycollect</span><span class="params">(node *Node, partial []<span class="keyword">rune</span>)</span> []<span class="title">string</span></span> {</span><br><span class="line"> <span class="keyword">var</span> (</span><br><span class="line"> m <span class="keyword">uint64</span></span><br><span class="line"> i <span class="keyword">int</span></span><br><span class="line"> p potentialSubtree</span><br><span class="line"> keys []<span class="keyword">string</span></span><br><span class="line"> )</span><br><span class="line"></span><br><span class="line"> potential := []potentialSubtree{potentialSubtree{<span class="number">0</span>, node}}</span><br><span class="line"> <span class="keyword">for</span> l := <span class="built_in">len</span>(potential); l > <span class="number">0</span>; l = <span class="built_in">len</span>(potential) {</span><br><span class="line"> i := l - <span class="number">1</span></span><br><span class="line"> p = potential[i]</span><br><span class="line"> potential = potential[:i]</span><br><span class="line"> m = maskruneslice(partial[p.idx:])</span><br><span class="line"> <span class="keyword">if</span> (p.node.mask & m) != m {</span><br><span class="line"> <span class="keyword">continue</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> p.node.val == partial[p.idx] {</span><br><span class="line"> p.idx++</span><br><span class="line"> <span class="keyword">if</span> p.idx == <span class="built_in">len</span>(partial) {</span><br><span class="line"> keys = <span class="built_in">append</span>(keys, collect(p.node)...)</span><br><span class="line"> <span class="keyword">continue</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> _, c := <span class="keyword">range</span> p.node.children {</span><br><span class="line"> potential = <span class="built_in">append</span>(potential, potentialSubtree{node: c, idx: p.idx})</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> keys</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="search"><a href="#search" class="headerlink" title="search"></a>search</h2><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> ByKeys []<span class="keyword">string</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(a ByKeys)</span> <span class="title">Len</span><span class="params">()</span> <span class="title">int</span></span> { <span class="keyword">return</span> <span class="built_in">len</span>(a) }</span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(a ByKeys)</span> <span class="title">Swap</span><span class="params">(i, j <span class="keyword">int</span>)</span></span> { a[i], a[j] = a[j], a[i] }</span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(a ByKeys)</span> <span class="title">Less</span><span class="params">(i, j <span class="keyword">int</span>)</span> <span class="title">bool</span></span> { <span class="keyword">return</span> <span class="built_in">len</span>(a[i]) < <span class="built_in">len</span>(a[j]) }</span><br></pre></td></tr></table></figure><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(t *Trie)</span> <span class="title">HasKeysWithPrefix</span><span class="params">(key <span class="keyword">string</span>)</span> <span class="title">bool</span></span> {</span><br><span class="line"> node := findNode(t.Root(), []<span class="keyword">rune</span>(key))</span><br><span class="line"> <span class="keyword">return</span> node != <span class="literal">nil</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(t Trie)</span> <span class="title">FuzzySearch</span><span class="params">(pre <span class="keyword">string</span>)</span> []<span class="title">string</span></span> {</span><br><span class="line"> keys := fuzzycollect(t.Root(), []<span class="keyword">rune</span>(pre))</span><br><span class="line"> sort.Sort(ByKeys(keys))</span><br><span class="line"> <span class="keyword">return</span> keys</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(t Trie)</span> <span class="title">PrefixSearch</span><span class="params">(pre <span class="keyword">string</span>)</span> []<span class="title">string</span></span> {</span><br><span class="line"> node := findNode(t.Root(), []<span class="keyword">rune</span>(pre))</span><br><span class="line"> <span class="keyword">if</span> node == <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> collect(node)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(t *Trie)</span> <span class="title">Keys</span><span class="params">()</span> []<span class="title">string</span></span> {</span><br><span class="line"> <span class="keyword">return</span> t.PrefixSearch(<span class="string">""</span>)</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>假设我们要实现一个类似于有道词典这样的字典，开始时使用哈希表来存储键值对，后来在实现前缀搜索，模糊匹配等相关功能时，发现使用哈希表作为底层存储的数据结构有一个致命的缺点，基于哈希表的前缀搜索，模糊匹配实现难度非常之大，所以决定换一种数据结构来存储数据。下面我们来介绍一下 <
</summary>
</entry>
<entry>
<title>Page Views</title>
<link href="https://oceanoverflow.github.io/2018/06/23/PageViews/"/>
<id>https://oceanoverflow.github.io/2018/06/23/PageViews/</id>
<published>2018-06-23T15:08:44.000Z</published>
<updated>2018-06-26T08:48:33.155Z</updated>
<content type="html"><![CDATA[<p>统计网站的页面浏览次数（ <code>Page Views</code> ）是分析网站的重要参数之一。由于多个用户发送的请求很可能是同时进行的，所以这里涉及到并发操作时对变量的保护问题。</p><h2 id="mutex"><a href="#mutex" class="headerlink" title="mutex"></a>mutex</h2><p>在统计页面浏览次数时，我们可以使用一个变量 <code>pageView</code> 来表示页面的浏览次数，并用一个 <code>mutex</code> 锁来保护变量存取。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> (</span><br><span class="line"> pageView <span class="keyword">int</span></span><br><span class="line"> mutex sync.Mutex</span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">handler</span><span class="params">(w http.ResponseWriter, r *http.Request)</span></span> {</span><br><span class="line"> mutex.Lock()</span><br><span class="line"> pageView++</span><br><span class="line"> mutex.Unlock()</span><br><span class="line"> fmt.Fprintf(w, <span class="string">"<h1>%v</h1>"</span>, pageView)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> http.HandleFunc(<span class="string">"/"</span>, handler)</span><br><span class="line"> http.ListenAndServe(<span class="string">"localhost:8080"</span>, <span class="literal">nil</span>)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="channel"><a href="#channel" class="headerlink" title="channel"></a>channel</h2><p>除了使用互斥锁来保护变量之外，我们来可以使用 <code>golang</code> 中特有的<code>unbuffered channel</code> 来获取页面浏览次数，我们在 <code>main</code> 函数中起一个 <code>goroutine</code> 来生产递增的 <code>nextID</code> ，每个 <code>nextID <- i</code> 发送操作均会阻塞直到一个 <code>goroutine</code> 启动时通过 <code><-nextID</code> 来接收，这里因为使用了<code>unbuffered channel</code> ，所以保证了整个过程的原子性。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> nextID = <span class="built_in">make</span>(<span class="keyword">chan</span> <span class="keyword">int</span>)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">handler</span><span class="params">(w http.ResponseWriter, r *http.Request)</span></span> {</span><br><span class="line"> fmt.Fprintf(w, <span class="string">"<h1>%v</h1>"</span>, <-nextID)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> http.HandleFunc(“/”, handler)</span><br><span class="line"> <span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line"> <span class="keyword">for</span> i := <span class="number">0</span>; ; i++ {</span><br><span class="line"> nextID <- i</span><br><span class="line"> }</span><br><span class="line"> }()</span><br><span class="line"> http.ListenAndServe(<span class="string">"localhost:8080"</span>, <span class="literal">nil</span>)</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>统计网站的页面浏览次数（ <code>Page Views</code> ）是分析网站的重要参数之一。由于多个用户发送的请求很可能是同时进行的，所以这里涉及到并发操作时对变量的保护问题。</p>
<h2 id="mutex"><a href="#mutex" class="
</summary>
</entry>
<entry>
<title>Skip List</title>
<link href="https://oceanoverflow.github.io/2018/06/21/SkipList/"/>
<id>https://oceanoverflow.github.io/2018/06/21/SkipList/</id>
<published>2018-06-21T12:53:24.000Z</published>
<updated>2018-06-26T08:47:54.101Z</updated>
<content type="html"><![CDATA[<p>跳表是一种建立在普通链表基础上的概率数据结构，为什么说是概率呢，因为跳表会根据概率在原始链表上构建后续的链表层，每个额外的链表层包含的元素数量更少，但每个元素都是原始链表中已经拥有的，这使得跳表在搜索时不用像链表一样一个一个元素的寻找，而是跳过一些元素，这就是跳表名字的由来，正是因为略过了一些元素，所以大大增加了搜索的速度。</p><p><img src="https://ds055uzetaobb.cloudfront.net/image_optimizer/968e3d82130f4bfa9a1574b1537c840a1787c919.png" alt="SkipList"></p><p>利用跳表可以构建 <code>kv</code> 数据库的索引，例如 <code>leveldb</code> 中的索引就是利用跳表来构建的，跳表相对其他索引数据结构例如红黑树，b树等有实现相对简单，但查询效率高的特点，其空间复杂度为 <code>O(n)</code> ，时间复杂度为 <code>O(logn)</code> 。</p><h2 id="跳表的组成元素"><a href="#跳表的组成元素" class="headerlink" title="跳表的组成元素"></a>跳表的组成元素</h2><p>与链表类似，跳表也是由一系列节点组成，每个节点我们用 <code>Element</code> 结构体表示，每个节点会存储一个 <code>kv</code> 对， 除了 <code>kv</code> 对之外，还包含一个 <code>elementNode</code> 结构体，该结构体其实就是一个指针数组，在单向链表中，指针有且只有一个，在跳表中，可以存在多个指向后续节点的指针，用于遍历时加速跳跃使用，这个指针数组是跳表的核心，使用 <code>next []*Element</code> 表示。</p><p>在表示完跳表的基本元素之后，我们还需要考虑一个完整跳表的数据结构，类与普通的链表类似，我们需要一个跳表头，这里用 <code>elementNode</code> 表示，由于跳表在构建时新元素的高度是不确定的，所以我们需要一个 <code>rand.Source</code> 帮助我们决定元素的高度，并限制高度小于最高高度 <code>maxLevel</code> ，这里整个跳表的质量与随机性有相当大的关系，构建得到的跳表随机性越强，搜索速度越快。</p><p>还有我们需要 <code>prevNodesCache []*elementNode</code> 来存储被查找节点的之前的节点，这个刚开始理解起来有一点复杂，因为被查找节点每一层的前一个元素都是不同的，每一个之前的元素可以用一个 <code>*elementNode</code> 来表示，由于每个节点可能存在多层指针，所以这里用一个数组来表示 <code>[]*elementNode</code> 。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> elementNode <span class="keyword">struct</span> {</span><br><span class="line"> next []*Element</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> Element <span class="keyword">struct</span> {</span><br><span class="line"> elementNode</span><br><span class="line"> key <span class="keyword">float64</span></span><br><span class="line"> value <span class="keyword">interface</span>{}</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> SkipList <span class="keyword">struct</span> {</span><br><span class="line"> elementNode</span><br><span class="line"> maxLevel <span class="keyword">int</span></span><br><span class="line"> Length <span class="keyword">int</span></span><br><span class="line"> randSource rand.Source</span><br><span class="line"> mutex sync.RWMutex</span><br><span class="line"> prevNodesCache []*elementNode</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">New</span><span class="params">(maxLevel <span class="keyword">int</span>)</span> *<span class="title">SkipList</span></span> {</span><br><span class="line"> <span class="keyword">if</span> maxLevel < <span class="number">1</span> || maxLevel > <span class="number">64</span> {</span><br><span class="line"> <span class="built_in">panic</span>(<span class="string">"max level should be less than 64 and greater than 1"</span>)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> &SkipList{</span><br><span class="line"> elementNode: elementNode{next: <span class="built_in">make</span>([]*Element, maxLevel)},</span><br><span class="line"> maxLevel: maxLevel,</span><br><span class="line"> randSource: rand.New(rand.NewSource(time.Now().UnixNano())),</span><br><span class="line"> prevNodesCache: <span class="built_in">make</span>([]*elementNode, maxLevel),</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="跳表的基本操作"><a href="#跳表的基本操作" class="headerlink" title="跳表的基本操作"></a>跳表的基本操作</h2><p>在插入和删除特定节点时，我们需要获取该节点之前的所有元素节点， <code>getPrevElementNodes</code> 从跳表的头节点（ <code>list.elementNode</code> ）出发，并从最高层开始，确定 <code>key</code> 对应的元素在该层对应的前一个<code>elementNode</code> ，然后存储在 <code>prevNodesCache</code> 中（ <code>prevs[i] = prev</code> ），这样一直查找直到最底层。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(list *SkipList)</span> <span class="title">getPrevElementNodes</span><span class="params">(key <span class="keyword">float64</span>)</span> []*<span class="title">elementNode</span></span> {</span><br><span class="line"> <span class="keyword">var</span> prev *elementNode = &list.elementNode</span><br><span class="line"> <span class="keyword">var</span> next *Element</span><br><span class="line"></span><br><span class="line"> prevs := list.prevNodesCache</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> i := list.maxLevel - <span class="number">1</span>; i >=<span class="number">0</span>; i-- {</span><br><span class="line"> next = prev.next[i]</span><br><span class="line"> <span class="keyword">for</span> next != <span class="literal">nil</span> && key > next.key {</span><br><span class="line"> prev = &next.elementNode</span><br><span class="line"> next = next.next[i]</span><br><span class="line"> }</span><br><span class="line"> prevs[i] = prev</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> prevs</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(list *SkipList)</span> <span class="title">randLevel</span><span class="params">()</span> <span class="params">(level <span class="keyword">int</span>)</span></span> {</span><br><span class="line"> <span class="keyword">return</span> list.randSource.Intn(list.maxLevel)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="Get"><a href="#Get" class="headerlink" title="Get"></a>Get</h3><p><code>Get</code> 方法的本质就是一个走楼梯的过程，但是这里每个阶梯的高度和宽度并非确定，开始从楼梯的最高层出发（ <code>i := list.maxLevel - 1</code> ），每次对前方同一层的阶梯（ <code>next = prev.next[i]</code> ）进行试探，如果前方阶梯不为空（ <code>next != nil</code> ），且符合特定要求（ <code>key > next.key</code> ）则继续在这一层走，否则直接进入下一层（ <code>i--</code> ），直到找到为止，最后判断此阶梯不为空（ <code>next != nil</code> ）且该阶梯保存的值和要找的值相同时，返回该阶梯。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(list *SkipList)</span> <span class="title">Get</span><span class="params">(key <span class="keyword">float64</span>)</span> *<span class="title">Element</span></span> {</span><br><span class="line"> list.mutex.Lock()</span><br><span class="line"> <span class="keyword">defer</span> list.mutex.Unlock()</span><br><span class="line"></span><br><span class="line"> <span class="keyword">var</span> prev *elementNode = &list.elementNode</span><br><span class="line"> <span class="keyword">var</span> next *Element</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> i := list.maxLevel - <span class="number">1</span>; i >= <span class="number">0</span>; i-- {</span><br><span class="line"> next = prev.next[i]</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> next != <span class="literal">nil</span> && key > next.key {</span><br><span class="line"> prev = &next.elementNode</span><br><span class="line"> next = next.next[i]</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> next != <span class="literal">nil</span> && next.key <= key {</span><br><span class="line"> <span class="keyword">return</span> next</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="Set"><a href="#Set" class="headerlink" title="Set"></a>Set</h3><p><code>Set</code> 方法需要使用之前的 <code>getPrevElementNodes</code> 来获取对应 <code>key</code> 之前的元素，如果该元素对应的 <code>key</code> 已经存在，则改变其 <code>value</code> 之后返回，如果不存在，则构建一个新的节点，然后将自己之前的节点的指针和自己之后节点的指针相连，使其前节点指向自己，使自己指向后面的节点。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(list *SkipList)</span> <span class="title">Set</span><span class="params">(key <span class="keyword">float64</span>, value <span class="keyword">interface</span>{})</span> *<span class="title">Element</span></span> {</span><br><span class="line"> list.mutex.Lock()</span><br><span class="line"> <span class="keyword">defer</span> list.mutex.Unlock()</span><br><span class="line"></span><br><span class="line"> <span class="keyword">var</span> element *Element</span><br><span class="line"> prevs := list.getPrevElementNodes(key)</span><br><span class="line"> <span class="keyword">if</span> element = prevs[<span class="number">0</span>].next[<span class="number">0</span>]; element != <span class="literal">nil</span> && element.key <= key {</span><br><span class="line"> element.value = value</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"> }</span><br><span class="line"> element = &Element {</span><br><span class="line"> elementNode: elementNode{</span><br><span class="line"> next: <span class="built_in">make</span>([]*Element, list.randLevel()),</span><br><span class="line"> key: key,</span><br><span class="line"> value: value,</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> i := <span class="keyword">range</span> element.next {</span><br><span class="line"> element.next[i] = prevs[i].next[i]</span><br><span class="line"> prevs[i].next[i] = element</span><br><span class="line"> }</span><br><span class="line"> list.Length++</span><br><span class="line"> <span class="keyword">return</span> element</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="Remove"><a href="#Remove" class="headerlink" title="Remove"></a>Remove</h3><p><code>Remove</code> 方法也需要使用 <code>getPrevElementNodes</code> 方法，使自己前面的节点不指向该元素而是指向该元素后面的元素。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(list *SkipList)</span> <span class="title">Remove</span><span class="params">(key <span class="keyword">float64</span>)</span> *<span class="title">Element</span></span> {</span><br><span class="line"> list.mutex.Lock()</span><br><span class="line"> <span class="keyword">defer</span> list.mutex.Unlock()</span><br><span class="line"> prevs := list.getPrevElementNodes(key)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> element := prevs[<span class="number">0</span>].next[<span class="number">0</span>]; element != <span class="literal">nil</span> && element.key <= key {</span><br><span class="line"> <span class="keyword">for</span> k, v := <span class="keyword">range</span> element.next {</span><br><span class="line"> prevs[k].next[k] = v</span><br><span class="line"> }</span><br><span class="line"> list.Length--</span><br><span class="line"> <span class="keyword">return</span> element</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>跳表是一种建立在普通链表基础上的概率数据结构，为什么说是概率呢，因为跳表会根据概率在原始链表上构建后续的链表层，每个额外的链表层包含的元素数量更少，但每个元素都是原始链表中已经拥有的，这使得跳表在搜索时不用像链表一样一个一个元素的寻找，而是跳过一些元素，这就是跳表名字的由来
</summary>
</entry>
<entry>
<title>grep</title>
<link href="https://oceanoverflow.github.io/2018/06/20/grep/"/>
<id>https://oceanoverflow.github.io/2018/06/20/grep/</id>
<published>2018-06-20T06:07:24.000Z</published>
<updated>2018-06-26T08:45:51.416Z</updated>
<content type="html"><![CDATA[<p>最近看到 <code>golang</code> 语法中一个非常有趣的基本组合 <code>chan chan</code> ，为了加深自己的理解，想多看几个使用案例，于是去 <code>$GOPATH</code> 下面的文件去找，但是一个个文件找肯定不现实，于是借助 <code>grep</code> 帮我们进行搜索，<code>grep</code> 的使用非常简单，但是如果在不了解 <code>grep</code> 使用的情况下，自己实现一个类似的搜索功能应该也不是很难，今天就让我们来看看 <code>grep</code> 的使用和 <code>golang</code> 版本的简单实现吧。</p><h2 id="usr-bin-grep"><a href="#usr-bin-grep" class="headerlink" title="/usr/bin/grep"></a>/usr/bin/grep</h2><p><code>grep</code> 命令的基本语法如下所示。</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ grep <span class="string">'chan'</span> file</span><br><span class="line">$ grep <span class="string">'chan chan'</span> file1 file2</span><br><span class="line">$ grep [option] <span class="string">'chan chan'</span> file1 file2</span><br></pre></td></tr></table></figure><p>除了列出文件名之外，我们还可以使用通配符搜索当前目录中所有文件。</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ grep <span class="string">'chan chan'</span> *</span><br></pre></td></tr></table></figure><h3 id="search-sub-directories-recursively-using-grep"><a href="#search-sub-directories-recursively-using-grep" class="headerlink" title="search sub directories recursively using grep"></a>search sub directories recursively using grep</h3><p>使用 <code>grep</code> 加上 <code>-r</code> 或者 <code>-R</code> 选项可以递归搜索当前目录下的所有文件。</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ grep -r <span class="string">'chan chan'</span> *</span><br><span class="line">$ grep -R <span class="string">'chan chan'</span> *</span><br></pre></td></tr></table></figure><h3 id="examples"><a href="#examples" class="headerlink" title="examples"></a>examples</h3><p>例如，以下命令将会输出当前目录中所有文件及其所有子目录中包含单词 <code>chan chan</code> 的每一行。</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ grep -r <span class="string">'chan chan'</span> *</span><br></pre></td></tr></table></figure><h2 id="my-own-grep"><a href="#my-own-grep" class="headerlink" title="my own grep"></a>my own grep</h2><p>如果要自己实现，在不考虑性能的情况下， 只需要实现两个功能，一是文件夹的递归遍历，二就是模式匹配，下面我们实现一个最简单的 <code>grep</code> 。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> (</span><br><span class="line"> haystack = flag.String(<span class="string">"path"</span>, <span class="string">""</span>, <span class="string">"file path to search in"</span>)</span><br><span class="line"> needle = flag.String(<span class="string">"search"</span>, <span class="string">""</span>, <span class="string">"search string to look for"</span>)</span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> flag.Parse()</span><br><span class="line"> fi, err := os.Stat(*haystack)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> log.Fatal(err)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> fi.Mode().IsDir() {</span><br><span class="line"> *haystack = strings.TrimRight(*haystack, <span class="string">"/"</span>) + <span class="string">"/"</span></span><br><span class="line"> dfs(*haystack, *needle)</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> log.Fatal(<span class="string">"path must be a dir"</span>)</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="depth-first-search"><a href="#depth-first-search" class="headerlink" title="depth first search"></a>depth first search</h3><p>我们利用深度优先搜索来遍历相关文件，如果当前对应的 <code>path</code> 为目录（ <code>file.Mode().IsDir()</code> ），则递归调用搜索方法，如果 <code>path</code> 为普通文件（ <code>file.Mode().IsRegular()</code> ），则利用下面的 <code>grep</code> 方法扫描该文件。直到所有文件均被遍历。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">dfs</span><span class="params">(path, search <span class="keyword">string</span>)</span></span> {</span><br><span class="line"> files, err := ioutil.ReadDir(path)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> fmt.Println(err)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> _, file := <span class="keyword">range</span> files {</span><br><span class="line"> fullpath := path + file.Name(<span class="number">0</span></span><br><span class="line"> <span class="keyword">if</span> file.Mode().IsDir() {</span><br><span class="line"> dfs(fullpath+<span class="string">"/"</span>, search)</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> file.Mode().IsRegular() {</span><br><span class="line"> grep(fullpath, search)</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="grep-individual-using-scanner"><a href="#grep-individual-using-scanner" class="headerlink" title="grep individual using scanner"></a>grep individual using scanner</h3><p>对于每一个 <code>fullpath</code> 对应的文件，我们利用 <code>bufio.NewScanner</code> 构建一个扫描仪，并进行逐行扫描，然后判断要搜索的文字（ <code>search</code> ）是否存在于该行中，如果存在（ <code>strings.Contains</code> ），则在命令行中输出。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">grep</span><span class="params">(fullpath, search <span class="keyword">string</span>)</span></span> {</span><br><span class="line"> f, err := os.Open(fullpath)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> fmt.Println(err)</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"> }</span><br><span class="line"> scanner := bufio.NewScanner(f)</span><br><span class="line"> <span class="keyword">for</span> scanner.Scan() {</span><br><span class="line"> line := scanner.Text()</span><br><span class="line"> <span class="keyword">if</span> strings.Contains(line, search) {</span><br><span class="line"> fmt.Println(fullpath, <span class="string">" "</span>, line)</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>最近看到 <code>golang</code> 语法中一个非常有趣的基本组合 <code>chan chan</code> ，为了加深自己的理解，想多看几个使用案例，于是去 <code>$GOPATH</code> 下面的文件去找，但是一个个文件找肯定不现实，于是借助 <
</summary>
</entry>
<entry>
<title>Pipeline</title>
<link href="https://oceanoverflow.github.io/2018/06/19/Pipeline/"/>
<id>https://oceanoverflow.github.io/2018/06/19/Pipeline/</id>
<published>2018-06-19T08:39:47.000Z</published>
<updated>2018-06-26T08:48:31.242Z</updated>
<content type="html"><![CDATA[<p><img src="http://insideenergy.org/wp-content/uploads/2017/05/DAPL-Pipeline-2-1170x775.jpg" alt="pipeline"></p><p>在介绍 <code>io.Pipe</code> 之前，我们先来看一段非常简单的代码，如果现在需要向服务器发送一个 <code>POST</code> 请求，按流程一步步来，首先构建一个消息结构体，然后将该结构体编码成 <code>json</code> 格式的数据，并将该数据暂时存储在 <code>buf</code> 中，最后利用 <code>http.Post</code> 发送该请求，轻松愉快。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> message <span class="keyword">struct</span> {</span><br><span class="line"> Name <span class="keyword">string</span></span><br><span class="line"> Age <span class="keyword">int</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> m := &message{Name: <span class="string">"yiyangyi"</span>, Age: <span class="number">23</span>}</span><br><span class="line"> <span class="keyword">var</span> buf bytes.Buffer</span><br><span class="line"> err := json.NewEncoder(&buf).Encode(m)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> log.Fatal(err)</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> resp, err := http.Post(<span class="string">"example.com"</span>, <span class="string">"application/json"</span>, &buf)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> log.Fatal(err) </span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">defer</span> resp.Body.Close()</span><br><span class="line"> </span><br><span class="line"> b, err := ioutil.ReadAll(resp.Body)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> log.Fatal(err)</span><br><span class="line"> }</span><br><span class="line"> fmt.Println(<span class="keyword">string</span>(b))</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>但是，上面的代码存在一个问题，就是我们需要分配一个缓存区（ <code>bytes.Buffer</code> ）存储编码后的数据，为了节省内存，能不能通过一个方法来避免此部分内存的分配呢，解决方法就是我们今天的主角——<code>io.Pipe</code> 。</p><p><code>io.Pipe</code> 执行后会得到一个 <code>PipeReader</code> 和 <code>PipeWriter</code> ，当程序向<code>PipeWriter</code> 中写入数据时，<code>PipeReader</code> 可以同步地收到这部分消息，根据这个特性，我们将 <code>PipeWriter</code> 代替 <code>bytes.Buffer</code> 作为参数 (它们同为 <code>io.Writer</code> ) 传入 <code>json.NewEncoder</code> 中，当我们 <code>Encode</code> 数据时，会直接向 <code>PipeWriter</code> 写入数据，此时 <code>Pipe</code> 的另一边，<code>PipeReader</code> 可以马上收到数据，利用 <code>io.Pipe</code> 我们就避免了中间非必要缓存的分配。</p><p>由于使用 <code>io.Pipe</code> 发送和接收数据是一个同步的过程 (类似于 <code>unbuffered channel</code> )，所以需要让发送和接收在两个 <code>goroutine</code> 中进行，如果它们存在于一个 <code>goroutine</code> 中，会导致程序死锁。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> message <span class="keyword">struct</span> {</span><br><span class="line"> Name <span class="keyword">string</span></span><br><span class="line"> Age <span class="keyword">int</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> pr, pw := io.Pipe()</span><br><span class="line"></span><br><span class="line"> <span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> { </span><br><span class="line"> m := &message{Name: <span class="string">"yiyangyi"</span>, Age: <span class="number">23</span>}</span><br><span class="line"> err := json.NewEncoder(pw).Encode(m)</span><br><span class="line"> pw.CloseWithError(err)</span><br><span class="line"> }()</span><br><span class="line"></span><br><span class="line"> resp, err := http.Post(<span class="string">"example.com"</span>, <span class="string">"application/json"</span>, pr)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> log.Fatal(err) </span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">defer</span> resp.Body.Close()</span><br><span class="line"></span><br><span class="line"> b, err := ioutil.ReadAll(resp.Body)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> log.Fatal(err)</span><br><span class="line"> }</span><br><span class="line"> fmt.Println(<span class="keyword">string</span>(b))</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="io-TeeReader-amp-io-MultiReader-❤️-io-Pipe"><a href="#io-TeeReader-amp-io-MultiReader-❤️-io-Pipe" class="headerlink" title="io.TeeReader & io.MultiReader ❤️ io.Pipe"></a>io.TeeReader & io.MultiReader ❤️ io.Pipe</h2><p><code>io.Pipe</code> 经常与 <code>io.TeeReader</code> 和 <code>io.MultiReader</code> 一起使用，下面有两个这样的例子。</p><h3 id="io-Pipe-❤️-io-TeeReader"><a href="#io-Pipe-❤️-io-TeeReader" class="headerlink" title="io.Pipe ❤️ io.TeeReader"></a>io.Pipe ❤️ io.TeeReader</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">tr <--- r -----> pw ------ > pr</span><br></pre></td></tr></table></figure><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">upload</span><span class="params">(r io.Reader)</span></span> {</span><br><span class="line"> pr, pw := io.Pipe()</span><br><span class="line"> tr := io.TeeReader(r, pw)</span><br><span class="line"></span><br><span class="line"> done := <span class="built_in">make</span>(<span class="keyword">chan</span> <span class="keyword">bool</span>)</span><br><span class="line"> <span class="keyword">defer</span> <span class="built_in">close</span>(done)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line"> <span class="keyword">defer</span> pw.Close()</span><br><span class="line"> uploadFile(tr)</span><br><span class="line"> done <- <span class="literal">true</span></span><br><span class="line"> }()</span><br><span class="line"></span><br><span class="line"> <span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line"> webmr := transcode(pr)</span><br><span class="line"> uploadFile(webmr)</span><br><span class="line"> done <- <span class="literal">true</span></span><br><span class="line"> }()</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> c := <span class="number">0</span>; c < <span class="number">2</span>; c++ {</span><br><span class="line"> <-done</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="io-Pipe-❤️-io-MultiReader"><a href="#io-Pipe-❤️-io-MultiReader" class="headerlink" title="io.Pipe ❤️ io.MultiReader"></a>io.Pipe ❤️ io.MultiReader</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><- pr1 <-----> pw1 --\</span><br><span class="line"></span><br><span class="line"><- pr2 <-----> pw2 ----</span><br><span class="line"> > multiwriter(pw1,pw2,pw3,pw4) <----- r</span><br><span class="line"><- pr3 <-----> pw3 ----</span><br><span class="line"></span><br><span class="line"><- pr4 <-----> pw4 --/</span><br></pre></td></tr></table></figure><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">TranscodeAll</span><span class="params">(r io.Reader)</span></span> {</span><br><span class="line"> mp4R, mp4W := io.Pipe()</span><br><span class="line"> oggR, oggW := io.Pipe()</span><br><span class="line"> wavR, wavW := io.Pipe()</span><br><span class="line"> webmR, webmW := io.Pipe()</span><br><span class="line"></span><br><span class="line"> done := <span class="built_in">make</span>(<span class="keyword">chan</span> <span class="keyword">bool</span>)</span><br><span class="line"> <span class="keyword">defer</span> <span class="built_in">close</span>(done)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">go</span> transcodeMP4(mp4R, done)</span><br><span class="line"> <span class="keyword">go</span> transcodeOgg(oggR, done)</span><br><span class="line"> <span class="keyword">go</span> transcodeWav(wavR, done)</span><br><span class="line"> <span class="keyword">go</span> transcodeWebM(webmR, done)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line"> <span class="keyword">defer</span> mp4W.Close() </span><br><span class="line"> <span class="keyword">defer</span> oggW.Close()</span><br><span class="line"> <span class="keyword">defer</span> wavW.Close()</span><br><span class="line"> <span class="keyword">defer</span> webmW.Close()</span><br><span class="line"></span><br><span class="line"> mw := io.MultiWriter(mp4W, oggW, wavW, webmW)</span><br><span class="line"> io.Copy(mw, r)</span><br><span class="line"> }()</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> c := <span class="number">0</span>; c < <span class="number">4</span>; c++ {</span><br><span class="line"> <-done</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p><img src="http://insideenergy.org/wp-content/uploads/2017/05/DAPL-Pipeline-2-1170x775.jpg" alt="pipeline"></p>
<p>在介绍 <code>io.Pipe</code
</summary>
</entry>
<entry>
<title>imgcat</title>
<link href="https://oceanoverflow.github.io/2018/06/18/imgcat/"/>
<id>https://oceanoverflow.github.io/2018/06/18/imgcat/</id>
<published>2018-06-18T14:02:51.000Z</published>
<updated>2018-06-26T08:46:17.739Z</updated>
<content type="html"><![CDATA[<p>在 <code>iTerm2</code> 中我们可以直接查看图片，是不是有点酷炫。</p><p><img src="https://www.iterm2.com/images/inline_image_sparky_demo.png" alt="imgcat"></p><p>为了使图片能够正确的展示，我们需要使数据满足特定的协议格式。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ESC ] 1337 ; File = [optional arguments] : base-64 encoded file contents ^G</span><br></pre></td></tr></table></figure><p>按照上面的格式我们将数据分为 <code>header</code> ，<code>body</code> 和 <code>footer</code> 三个部分，而 <code>header</code> 和 <code>footer</code> 相对固定，我们使用 <code>strings.NewReader</code> 封装返回一个 <code>io.Reader</code> ，而中间变化的 <code>base64</code> 编码的数据，我们需要利用 <code>base64.NewEncoder</code> 来对输入的数据进行编码，最后将三个部分利用 <code>io.MultiReader</code> 整合成一个 <code>io.Reader</code> ，然后利用 <code>io.Copy</code> 将数据传输到 <code>w</code> 中，然而 <code>base64.NewEncoder</code> 返回的是一个 <code>io.WriteCloser</code> ，而我们想要的是一个 <code>io.Reader</code> ，这种转化我们可以借助 <code>io.Pipe()</code> 实现，<code>io.Pipe()</code> 会返回一个 <code>PipeReader</code> 和一个 <code>PipeWriter</code> ，当有数据往 <code>PipeWriter</code> 中写时，<code>PipeReader</code> 就可以获得相应的数据，通过这种方式，我们就可以间接地将一个 <code>Writer</code> 转换为一个 <code>Reader</code> 来使用了。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">copy</span><span class="params">(w io.Writer, r io.Reader)</span> <span class="title">error</span></span> {</span><br><span class="line"> header := strings.NewReader(<span class="string">"\033 ] 1337 ; File = inline=1 : "</span>)</span><br><span class="line"> footer := strings.NewReader(<span class="string">"\a\n"</span>)</span><br><span class="line"> </span><br><span class="line"> pr, pw := io.Pipe()</span><br><span class="line"> <span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line"> <span class="keyword">defer</span> pw.Close()</span><br><span class="line"> wc := base64.NewEncoder(base64.StdEncoding, pw)</span><br><span class="line"> _, err := io.Copy(wc, r)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> pw.CloseWithError(err)</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">if</span> err := wc.Close(); err != <span class="literal">nil</span> {</span><br><span class="line"> pw.CloseWithError(err)</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"> }</span><br><span class="line"> }()</span><br><span class="line"> </span><br><span class="line"> mr := io.MultiReader(header, pr, footer)</span><br><span class="line"> _, err := io.Copy(w, mr)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> err</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>为了在下面的 <code>imgcat</code> 方法中使用 <code>io.Copy</code> 接口，我们需要自己实现一个 <code>io.Writer</code> ，在 <code>NewWriter</code> 函数中我们返回自己 <code>writer</code> 结构体，并且同时起了一个<code>goroutine</code> 来获取 <code>PipeWriter</code> 写的数据，同理，<code>PipeWriter</code> 中写入的数据可以从相应的 <code>PipeReader</code> 获取。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> imgcat</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> writer <span class="keyword">struct</span> {</span><br><span class="line"> pw *io.PipeWriter</span><br><span class="line"> done <span class="keyword">chan</span> <span class="keyword">struct</span>{}</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(w *writer)</span> <span class="title">Write</span><span class="params">(p []<span class="keyword">byte</span>)</span> <span class="params">(<span class="keyword">int</span>, error)</span></span> {</span><br><span class="line"> <span class="keyword">return</span> w.pw.Write(p)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(w *writer)</span> <span class="title">Close</span><span class="params">()</span> <span class="title">error</span></span> {</span><br><span class="line"> <span class="keyword">if</span> err := w.pw.Close(); err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> err</span><br><span class="line"> }</span><br><span class="line"> <-w.done</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">NewWriter</span><span class="params">(w io.Writer)</span> <span class="title">io</span>.<span class="title">WriteCloser</span></span> {</span><br><span class="line"> pr, pw := io.Pipe()</span><br><span class="line"> wc := &writer{pw, <span class="built_in">make</span>(<span class="keyword">chan</span> <span class="keyword">struct</span>{})}</span><br><span class="line"> <span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line"> <span class="keyword">defer</span> <span class="built_in">close</span>(w.done)</span><br><span class="line"> err := <span class="built_in">copy</span>(w, pr)</span><br><span class="line"> pr.CloseWithError(err)</span><br><span class="line"> }()</span><br><span class="line"> <span class="keyword">return</span> wc</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>在 <code>imgcat</code> 方法中主要是通过 <code>io.Copy</code> 将路径 <code>path</code> 对应的文件拷贝到我们上面的 <code>Writer</code> 中，<code>NewWriter</code> 指定 <code>os.Stdout</code> 作为我们的输出。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">imgcat</span><span class="params">(path <span class="keyword">string</span>)</span> <span class="title">error</span></span> {</span><br><span class="line"> f, err := os.Open(path)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> err</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">defer</span> f.Close()</span><br><span class="line"></span><br><span class="line"> wc := imgcat.NewWriter(os.Stdout)</span><br><span class="line"> <span class="keyword">if</span> _, err = io.Copy(wc, f); err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> err</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> wc.Close()</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> <span class="keyword">if</span> <span class="built_in">len</span>(os.Args) != <span class="number">2</span> {</span><br><span class="line"> <span class="keyword">return</span> </span><br><span class="line"> }</span><br><span class="line"> err := imgcat(os.Args[<span class="number">1</span>])</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> fmt.Println(err)</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>在 <code>iTerm2</code> 中我们可以直接查看图片，是不是有点酷炫。</p>
<p><img src="https://www.iterm2.com/images/inline_image_sparky_demo.png" alt="imgcat"></p>
</summary>
</entry>
<entry>
<title>World Cup</title>
<link href="https://oceanoverflow.github.io/2018/06/17/WorldCup/"/>
<id>https://oceanoverflow.github.io/2018/06/17/WorldCup/</id>
<published>2018-06-17T07:18:49.000Z</published>
<updated>2018-06-26T08:47:20.459Z</updated>
<content type="html"><![CDATA[<p>四年一度的世界杯最近正在俄罗斯如火如荼的进行，昨天看了西班牙🇪🇸对葡萄牙🇵🇹的比赛，C罗真的是太帅了，帽子戏法，不过作为一个伪球迷-真程序员，准备写一点和世界杯有关的东西，想想用机器学习预测比赛结果好像并没有什么卵用，于是就写一个最简单的搬砖程序来提醒自己比赛时间吧。最终效果如下所示，典型的 <code>MVC</code> 。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">Costa Rica <span class="number">0</span> - <span class="number">0</span> Serbia</span><br><span class="line">------------------------------------------------------------------------</span><br><span class="line">⚽ Will be played <span class="number">4</span> hours from now</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">Germany <span class="number">0</span> - <span class="number">0</span> Mexico</span><br><span class="line">------------------------------------------------------------------------</span><br><span class="line">⚽ Will be played <span class="number">7</span> hours from now</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">Brazil <span class="number">0</span> - <span class="number">0</span> Switzerland</span><br><span class="line">------------------------------------------------------------------------</span><br><span class="line">⚽ Will be played <span class="number">10</span> hours from now</span><br></pre></td></tr></table></figure><h2 id="Model"><a href="#Model" class="headerlink" title="Model"></a>Model</h2><p>比赛相关的数据可以从 <code>http://worldcup.sfg.io/</code> 获取，我们使用结构体定义数据模型，注意结构体中的数据字段必须为大写，否则无法正确 <code>Unmarshal</code>，如果想要使结构体中的字段与 <code>json</code> 中的字段不同，可以额外增加 <code>tag</code> 来说明。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> match <span class="keyword">struct</span> {</span><br><span class="line"> Time <span class="keyword">string</span> <span class="string">`json:"time,omitempty"`</span></span><br><span class="line"> DateTime <span class="keyword">string</span> <span class="string">`json:"datetime,omitempty"`</span></span><br><span class="line"> HomeTeam team <span class="string">`json:"home_team,ommitempty"`</span></span><br><span class="line"> AwayTeam team <span class="string">`json:"away_team,omitempty"`</span></span><br><span class="line"> Winner <span class="keyword">string</span> <span class="string">`json:"winner,omitempty"`</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> team <span class="keyword">struct</span> {</span><br><span class="line"> Country <span class="keyword">string</span> <span class="string">`json:"country,omitempty"`</span></span><br><span class="line"> Code <span class="keyword">string</span> <span class="string">`json:"code,omitempty"`</span></span><br><span class="line"> Goals <span class="keyword">int</span> <span class="string">`json:"goals,omitempty"`</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">fetch</span><span class="params">(endpoint)</span> <span class="params">([]match, error)</span></span> {</span><br><span class="line"> url := fmt.Sprintf(<span class="string">"http://worldcup.sfg.io/%s"</span>, endpoint)</span><br><span class="line"> res, err := http.Get(url)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span>, err</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">defer</span> res.Body.Close()</span><br><span class="line"> b, err := ioutil.ReadAll(res.Body)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span>, err</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">var</span> matches []match</span><br><span class="line"> err = json.Unmarshal(b, &matches)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span>, err</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> matches, <span class="literal">nil</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="View"><a href="#View" class="headerlink" title="View"></a>View</h2><p>因为我们的 <code>layout</code> 相对比较复杂，所以 <code>View</code> 部分代码也最多，但难度不大，就是繁琐，主要注意几点，彩色字体输出，不同情况下进度条的展示效果和各个字段所在的位置。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> status <span class="keyword">int</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> (</span><br><span class="line"> PAST status = <span class="literal">iota</span></span><br><span class="line"> NOW</span><br><span class="line"> FUTURE</span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> ScreenWidth = <span class="number">72</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> (</span><br><span class="line"> START = <span class="string">"\x1b[32m"</span></span><br><span class="line"> END = <span class="string">"\x1b[0m"</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">colorize</span><span class="params">(s <span class="keyword">string</span>)</span> <span class="title">string</span></span> {</span><br><span class="line"> <span class="keyword">return</span> START + s + END</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">multiply</span><span class="params">(c <span class="keyword">rune</span>, n <span class="keyword">int</span>)</span> <span class="title">string</span></span> {</span><br><span class="line"> <span class="keyword">var</span> cs []<span class="keyword">rune</span></span><br><span class="line"> <span class="keyword">for</span> i := <span class="number">0</span>; i < n; i++ {</span><br><span class="line"> cs = <span class="built_in">append</span>(cs, c)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">string</span>(cs)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">prettify</span><span class="params">(m match)</span> <span class="title">string</span></span> {</span><br><span class="line"></span><br><span class="line"> t, _ := time.Parse(time.RFC3339, m.DateTime)</span><br><span class="line"> diff := <span class="keyword">int</span>(time.Now().Sub(t).Seconds())</span><br><span class="line"></span><br><span class="line"> <span class="keyword">var</span> status status</span><br><span class="line"> <span class="keyword">switch</span> {</span><br><span class="line"> <span class="keyword">case</span> diff > <span class="number">60</span>*<span class="number">90</span>:</span><br><span class="line"> status = PAST</span><br><span class="line"> <span class="keyword">case</span> diff < <span class="number">0</span>:</span><br><span class="line"> status = FUTURE</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> status = NOW</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> percentage := <span class="number">0</span></span><br><span class="line"> matchStatus := <span class="string">""</span></span><br><span class="line"> on := <span class="literal">false</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">switch</span> status { </span><br><span class="line"> <span class="keyword">case</span> PAST:</span><br><span class="line"> percentage = <span class="number">100</span></span><br><span class="line"> <span class="keyword">var</span> result <span class="keyword">string</span></span><br><span class="line"> <span class="keyword">if</span> m.Winner == <span class="string">"Draw"</span> {</span><br><span class="line"> result = <span class="string">"Draw"</span></span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> result = fmt.Sprintf(<span class="string">"%s won"</span>, m.Winner)</span><br><span class="line"> }</span><br><span class="line"> matchStatus = fmt.Sprintf(<span class="string">"Played %s. %s"</span>, humanize.Time(t), result)</span><br><span class="line"> on = <span class="literal">true</span></span><br><span class="line"> <span class="keyword">case</span> NOW:</span><br><span class="line"> percentage = <span class="keyword">int</span>(diff / <span class="number">60</span> / <span class="number">90</span> * <span class="number">100</span>)</span><br><span class="line"> matchStatus = fmt.Sprintf(<span class="string">"Being played now: %s minutes gone"</span>, humanize.Time(t))</span><br><span class="line"> on = <span class="literal">true</span></span><br><span class="line"> <span class="keyword">case</span> FUTURE:</span><br><span class="line"> percentage = <span class="number">0</span></span><br><span class="line"> matchStatus = fmt.Sprintf(<span class="string">"Will be played %s"</span>, humanize.Time(t))</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> home := m.HomeTeam</span><br><span class="line"> away := m.AwayTeam</span><br><span class="line"></span><br><span class="line"> s, e := <span class="string">""</span>, <span class="string">""</span></span><br><span class="line"> <span class="keyword">if</span> on {</span><br><span class="line"> s = start</span><br><span class="line"> e = end</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> fmt.Sprintf(<span class="string">" %s%-32s %d - %d %32s%s\n %s\n \u26BD %s\n"</span>,</span><br><span class="line"> s,home.Country,home.Goals,away.Goals,away.Country,e,</span><br><span class="line"> progressBar(percentage),</span><br><span class="line"> matchStatus,</span><br><span class="line"> )</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">progressBar</span><span class="params">(percentage <span class="keyword">int</span>)</span> <span class="title">string</span></span> {</span><br><span class="line"> <span class="keyword">var</span> bar <span class="keyword">string</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">switch</span> percentage {</span><br><span class="line"> <span class="keyword">case</span> <span class="number">0</span>:</span><br><span class="line"> bar = multiply(<span class="string">'-'</span>, ScreenWidth)</span><br><span class="line"> <span class="keyword">case</span> <span class="number">100</span>:</span><br><span class="line"> bar = colorize(multiply(<span class="string">'-'</span>, ScreenWidth))</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> completed := <span class="keyword">int</span>(<span class="keyword">float64</span>(ScreenWidth) / <span class="number">100.0</span> * <span class="keyword">float64</span>(percentage))</span><br><span class="line"> bar = colorize(multiply(<span class="string">'-'</span>, completed<span class="number">-1</span>)+<span class="string">"o"</span>) + multiply(<span class="string">'-'</span>, ScreenWidth-completed)</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> bar</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="Controller"><a href="#Controller" class="headerlink" title="Controller"></a>Controller</h2><p><code>Controller</code> 部分就是将 <code>Model</code> 部分和 <code>view</code> 部分代码整合在一起。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> matches, err := fetch(<span class="string">"matches/today"</span>)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> log.Fatal(err)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> i, match := <span class="keyword">range</span> matches {</span><br><span class="line"> <span class="keyword">if</span> i == <span class="number">0</span> {</span><br><span class="line"> fmt.Println()</span><br><span class="line"> }</span><br><span class="line"> fmt.Println(prettify(match))</span><br><span class="line"> <span class="keyword">if</span> i != <span class="built_in">len</span>(matches)<span class="number">-1</span> {</span><br><span class="line"> fmt.Println()</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>四年一度的世界杯最近正在俄罗斯如火如荼的进行，昨天看了西班牙🇪🇸对葡萄牙🇵🇹的比赛，C罗真的是太帅了，帽子戏法，不过作为一个伪球迷-真程序员，准备写一点和世界杯有关的东西，想想用机器学习预测比赛结果好像并没有什么卵用，于是就写一个最简单的搬砖程序来提醒自己比赛时间吧
</summary>
</entry>
<entry>
<title>Snappy</title>
<link href="https://oceanoverflow.github.io/2018/06/15/Snappy/"/>
<id>https://oceanoverflow.github.io/2018/06/15/Snappy/</id>
<published>2018-06-15T09:00:01.000Z</published>
<updated>2018-06-26T08:47:49.515Z</updated>
<content type="html"><![CDATA[<p><code>Snappy</code> 是 <code>Google</code> 开发的压缩和解压缩的库，最初使用 <code>C++</code> 编写，后来又造了一个 <code>golang</code> 版本的轮子。由于目前该压缩算法还没有放到 <code>go</code> 的标准库中去，所以需要通过 <code>go get -u github.com/golang/snappy</code> 安装。</p><p>与其他常见的压缩算法相比，<code>Snappy</code> 存在的目的并不是为了获得最大程度的压缩比，正如其名字所言，它旨在保持合理的压缩比的同时最大限度地提高压缩和解压缩的速度。</p><p><code>Snappy</code> 对数据处理速度上做了很大的优化，但它并不是没有缺点，相对于其他算法来说，<code>Snappy</code> 压缩得到的文件体积会更大一点。如果你更在意压缩速度而不是压缩比的话，<code>Snappy</code> 可能会是一个很好的选择。</p><h2 id="compression"><a href="#compression" class="headerlink" title="compression"></a>compression</h2><p>下面，我们对比一下 <code>Snappy</code> 压缩算法和其他两种常见的压缩算法（ <code>zlib</code> 和 <code>gzip</code> ）的性能。首先，我们定一个 <code>Compressor</code> 接口，并使 <code>Zlib</code> ，<code>Gzip</code> 和 <code>Snappy</code> 这三个结构体均实现它。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> Zlib <span class="keyword">struct</span>{}</span><br><span class="line"><span class="keyword">type</span> Gzip <span class="keyword">struct</span>{}</span><br><span class="line"><span class="keyword">type</span> Snappy <span class="keyword">struct</span>{}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> Compressor <span class="keyword">interface</span> {</span><br><span class="line"> Compress(io.Writer, io.Reader) error</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="zlib"><a href="#zlib" class="headerlink" title="zlib"></a>zlib</h3><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(_ *Zlib)</span> <span class="title">Compress</span><span class="params">(w io.Writer, r io.Reader)</span> <span class="title">error</span></span> {</span><br><span class="line"> zw, err := zlib.NewWriterLevel(w, zlib.BestSpeed)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> err</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">defer</span> zw.Close()</span><br><span class="line"></span><br><span class="line"> _, err = io.Copy(zw, r)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> err</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="gzip"><a href="#gzip" class="headerlink" title="gzip"></a>gzip</h3><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(_ *Gzip)</span> <span class="title">Compress</span><span class="params">(w io.Writer, r io.Reader)</span> <span class="title">error</span></span> {</span><br><span class="line"> gw, err := gzip.NewWriterLevel(w, gzip.BestSpeed)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> err</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">defer</span> gw.Close()</span><br><span class="line"></span><br><span class="line"> _, err = io.Copy(gw, r)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> err</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="snappy"><a href="#snappy" class="headerlink" title="snappy"></a>snappy</h3><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(_ *Snappy)</span> <span class="title">compress</span><span class="params">(w io.Writer, r io.Reader)</span> <span class="title">error</span></span> {</span><br><span class="line"> sw := snappy.NewBufferedWriter(w)</span><br><span class="line"> <span class="keyword">defer</span> sw.Close()</span><br><span class="line"></span><br><span class="line"> _, err := io.Copy(sw, r)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> err</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="performance"><a href="#performance" class="headerlink" title="performance"></a>performance</h2><p>对于压缩的性能，可以从压缩比和压缩时间这两点进行衡量。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> f, err := os.Open(<span class="string">"somebigfile"</span>)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> log.Fatal(err)</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> fGzip, _ := os.Create(<span class="string">"gzip"</span>)</span><br><span class="line"> fZlib, _ := os.Create(<span class="string">"zlib"</span>)</span><br><span class="line"> fSnappy, _ := os.Create(<span class="string">"snappy"</span>)</span><br><span class="line"></span><br><span class="line"> cs := <span class="keyword">map</span>[compressor]io.Writer{</span><br><span class="line"> &Gzip{}: fGzip,</span><br><span class="line"> &Zlib{}: fZlib,</span><br><span class="line"> &Snappy{}: fSnappy,</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> c, v := <span class="keyword">range</span> cs {</span><br><span class="line"> t := time.Now()</span><br><span class="line"> err := c.compress(v, f)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> fmt.Println(err)</span><br><span class="line"> }</span><br><span class="line"> d := time.Since(t)</span><br><span class="line"> fmt.Printf(<span class="string">"method: %s, time spend: %.02f, ratio: %.05f%%\n"</span>, </span><br><span class="line"> v.Name(), </span><br><span class="line"> d.Seconds(), </span><br><span class="line"> compressionRatio(f, v))</span><br><span class="line"> f.Seek(<span class="number">0</span>, <span class="number">0</span>)</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">compressionRatio</span><span class="params">(f1, f2 *os.File)</span> <span class="title">float64</span></span> {</span><br><span class="line"> f1s, _ := f1.Stat()</span><br><span class="line"> f1size := f1s.Size()</span><br><span class="line"> f2s, _ := f2.Stat()</span><br><span class="line"> f2size := f2s.Size()</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">float64</span>(f2size) / <span class="keyword">float64</span>(f1size) * <span class="number">100</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>Snappy</code> 在速度上来说比 <code>zlib</code> 和 <code>gzip</code> 快得多，但文件相对要大 <code>20%</code> 到 <code>100%</code>。在 <code>64</code> 位模式的 <code>Core i7</code> 处理器上，<code>Snappy</code> 可以达到 <code>250~500</code> 兆每秒的压缩速度，可以说是非常惊人了。 </p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ go run main.go</span><br><span class="line">method: gzip, time spend: 8.38s, ratio: 7.576%</span><br><span class="line">method: zlib, time spend: 9.13s, ratio: 7.576%</span><br><span class="line">method: snappy, time spend: 1.96s, ratio: 18.813%</span><br></pre></td></tr></table></figure><p>相比其他的压缩库，<code>Snappy</code> 能在保持特定的压缩率下拥有惊人的压缩速度，压缩普通文本文件的速度是其他库的 <code>1.5-1.7</code> 倍，HTML能达到 <code>2-4</code> 倍，但是对于 <code>JPEG</code>、<code>PNG</code> 以及其他的已压缩的数据，压缩速度不会有明显改善。</p>]]></content>
<summary type="html">
<p><code>Snappy</code> 是 <code>Google</code> 开发的压缩和解压缩的库，最初使用 <code>C++</code> 编写，后来又造了一个 <code>golang</code> 版本的轮子。由于目前该压缩算法还没有放到 <code>go<
</summary>
</entry>
<entry>
<title>Timing Wheel</title>
<link href="https://oceanoverflow.github.io/2018/06/14/TimingWheel/"/>
<id>https://oceanoverflow.github.io/2018/06/14/TimingWheel/</id>
<published>2018-06-14T01:20:25.000Z</published>
<updated>2018-06-26T08:47:39.796Z</updated>
<content type="html"><![CDATA[<p><img src="https://www.confluent.io/wp-content/uploads/2016/08/TimingWheels1.png" alt="timingwheel"></p><p>当我们自己编写服务端程序时候，可能需要新建大量的定时器，对每个 <code>TCP</code> 连接设置连接超时，面对成千上万的客户端连接，如果傻傻地为每一个连接分配一个计时器的话，那么 <code>10k</code> 个连接就对应 <code>10k</code> 的计时器，<code>10w</code> 个连接就对应 <code>10w</code> 个计时器，这种方法显然是比较消耗内存的。</p><p>对于需要大量定时器的情况，由于许多定时器是在相近的时间超时的，或者说在一个时间范围内到时的，如果对时间精度没有那么近乎苛刻的要求的话，我们可以利用这个特性将多个同一时间段的计时器整合成一个来使用，这样的话就可以极大程度上降低内存的消耗，例如现在有 <code>3，4，5，6</code> 四个连接，它们均会在后面的3到4秒内超时，我们可以采用一种方法使它们共用一个计数器，并当时间点到来时，它们均可以被通知到。</p><h2 id="closed-channel"><a href="#closed-channel" class="headerlink" title="closed channel"></a>closed channel</h2><p>也就是说，我们需要一个机制来重复利用某一个计时器，换句话说，当时间点到来时，我们需要一种广播的机制让等待该计时器的都能被通知到。</p><p>为此需要利用 <code>golang</code> 中通道的几个特性，我们通过一个例子来理解一下，在下面的函数中 <code>make</code> 了一个 <code>unbuffered channel</code> ch，还新起了3个 <code>goroutine</code> ，每个<code>goroutine</code> 会执行 <code>fmt.Println("start ", i)</code> ，但等它们都执行到 <code><-ch</code> 语句时，由于没有其他 <code>goroutine</code> 执行 <code>ch <- struct{}{}</code> 操作，所以3个 <code>goroutine</code> 都会阻塞，当 <code>time.Sleep(1 * time.Second)</code> 结束时，我们利用 <code>close(ch)</code> ，此操作相当于向每个 <code>goroutine</code> 进行 <code>ch <- struct{}{}</code> ，此时阻塞解除，每个 <code>goroutine</code> 都可以继续执行 <code>fmt.Println("end ", i)</code> 。也就是说 <code>close(ch)</code> 相当于一个广播操作。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> ch := <span class="built_in">make</span>(<span class="keyword">chan</span> <span class="keyword">struct</span>{})</span><br><span class="line"> <span class="keyword">for</span> i := <span class="number">0</span>; i < <span class="number">3</span>; i++ {</span><br><span class="line"> i := i</span><br><span class="line"> <span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line"> fmt.Println(<span class="string">"start "</span>, i)</span><br><span class="line"> <-ch</span><br><span class="line"> fmt.Println(<span class="string">"end "</span>, i)</span><br><span class="line"> }()</span><br><span class="line"> }</span><br><span class="line"> time.Sleep(<span class="number">1</span> * time.Second)</span><br><span class="line"> <span class="built_in">close</span>(ch)</span><br><span class="line"> <span class="keyword">select</span> {}</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>上面的例子利用到了通道的的第一个特性和第三个特性。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">1. The expression blocks until a value is available. </span><br><span class="line">2. Receiving from a nil channel blocks forever. </span><br><span class="line">3. A receive operation on a closed channel can always proceed immediately, yielding the element type’s zero value after any previously sent values have been received.</span><br></pre></td></tr></table></figure><h2 id="timing-wheel"><a href="#timing-wheel" class="headerlink" title="timing wheel"></a>timing wheel</h2><p>那么上面的例子和我们要实现的东西有什么联系呢，其实联系非常大，我们可以给每一个需要进行定时操作的 <code>goroutine</code> 一个 <code>unbuffered channel</code><br>，并且给予在相同时间点超时的 <code>goroutine</code> 以同样的通道。然后我们自己维护一个 <code>ticker</code> ，这个 <code>ticker</code> 每隔固定的时间，将相应的通道关闭，这相当于一个广播操作，这样每个等待此广播的 <code>goroutine</code> 都会收到消息，也就可以继续执行下去了。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> TimingWheel <span class="keyword">struct</span> {</span><br><span class="line"> sync.Mutex</span><br><span class="line"></span><br><span class="line"> interval time.Duration</span><br><span class="line"> maxTimeout time.Duration </span><br><span class="line"> </span><br><span class="line"> ticker *time.Ticker</span><br><span class="line"> cs []<span class="keyword">chan</span> <span class="keyword">struct</span>{}</span><br><span class="line"> pos <span class="keyword">int</span></span><br><span class="line"></span><br><span class="line"> stop <span class="keyword">chan</span> <span class="keyword">struct</span>{}</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">New</span><span class="params">(interval time.Duration, buckets <span class="keyword">int</span>)</span> *<span class="title">TimingWheel</span></span> {</span><br><span class="line"> w := &TimingWheel{</span><br><span class="line"> interval: interval,</span><br><span class="line"> maxTimeout: interval * time.Duration(buckets),</span><br><span class="line"> ticker: time.NewTicker(interval),</span><br><span class="line"> cs : <span class="built_in">make</span>([]<span class="keyword">chan</span> <span class="keyword">struct</span>{}, buckets),</span><br><span class="line"> pos: <span class="number">0</span>,</span><br><span class="line"> stop: <span class="built_in">make</span>(<span class="keyword">chan</span> <span class="keyword">struct</span>{}),</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> i := <span class="keyword">range</span> w.cs {</span><br><span class="line"> w.cs[i] = <span class="built_in">make</span>(<span class="keyword">chan</span> <span class="keyword">struct</span>{})</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">go</span> w.run()</span><br><span class="line"> <span class="keyword">return</span> w</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>光说可能有一点抽象，下面我们来看一下具体的实现，<code>TimingWheel</code> 结构体相当于模拟一个秒表，其中的 <code>ticker</code> 就相当于一个秒针，同时我们还维护了一个通道数组 <code>cs</code> （其存储结构为顺序存储，而其逻辑结构则是一个圆环），作为定时器使用，<code>ticker</code> 每隔固定的 <code>interval</code> ，会让指针向前移动一个单位（ <code>pos++</code> ），并将该位置存储的通道关闭，也就相同于定时器返回，这种方法实现的定时器相对来说增加了定时器的利用率，降低了内存的消耗。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(w *TimingWheel)</span> <span class="title">run</span><span class="params">()</span></span> {</span><br><span class="line"> <span class="keyword">for</span> {</span><br><span class="line"> <span class="keyword">select</span> {</span><br><span class="line"> <span class="keyword">case</span> <-w.ticker.C:</span><br><span class="line"> w.tick()</span><br><span class="line"> <span class="keyword">case</span> <-w.stop:</span><br><span class="line"> w.ticker.Stop()</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(w *TimingWheel)</span> <span class="title">tick</span><span class="params">()</span></span> {</span><br><span class="line"> w.Lock()</span><br><span class="line"> last := w.cs[w.pos]</span><br><span class="line"> w.cs[w.pos] = <span class="built_in">make</span>(<span class="keyword">chan</span> <span class="keyword">struct</span>{})</span><br><span class="line"> </span><br><span class="line"> w.pos = (w.pos + <span class="number">1</span>) % <span class="built_in">len</span>(w.cs)</span><br><span class="line"> w.Unlock() </span><br><span class="line"> <span class="built_in">close</span>(last)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>After</code> 方法为我们提供一个定时器，如果 <code>timeout</code> 数值太大无法落在 <code>timing wheel</code> 的一个周期的时间范围内，则报错，如果 <code>timeout</code> 在正常范围内，则返回对应位置的通道，当该通道超时时，上面的 <code>tick</code> 方法就会将该通道关闭，相应的 <code>goroutine</code> 就可以收到通知了。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(w *TimingWheel)</span> <span class="title">After</span><span class="params">(timeout time.Duration)</span> <-<span class="title">chan</span> <span class="title">struct</span></span> {} {</span><br><span class="line"> <span class="keyword">if</span> timeout >= w.maxTimeout {</span><br><span class="line"> <span class="built_in">panic</span>(<span class="string">"timeout too long"</span>)</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> idx := <span class="keyword">int</span>(timeout / w.interval)</span><br><span class="line"> <span class="keyword">if</span> idx > <span class="number">0</span> {</span><br><span class="line"> idx--</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> w.Lock()</span><br><span class="line"> idx = (w.pos + idx) % <span class="built_in">len</span>(w.cs)</span><br><span class="line"> b := w.cs[idx]</span><br><span class="line"> w.Unlock()</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">return</span> b</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(w *TimingWheel)</span> <span class="title">Stop</span><span class="params">()</span></span> {</span><br><span class="line"> <span class="built_in">close</span>(w.exit)</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p><img src="https://www.confluent.io/wp-content/uploads/2016/08/TimingWheels1.png" alt="timingwheel"></p>
<p>当我们自己编写服务端程序时候，可能需要新建大量的定时器，对每
</summary>
</entry>
<entry>
<title>Ligatures</title>
<link href="https://oceanoverflow.github.io/2018/06/13/Ligatures/"/>
<id>https://oceanoverflow.github.io/2018/06/13/Ligatures/</id>
<published>2018-06-13T05:20:38.000Z</published>
<updated>2018-06-26T08:46:37.189Z</updated>
<content type="html"><![CDATA[<p>!!! From: <code>https://github.com/tonsky/FiraCode</code> !!!</p><h2 id="problem"><a href="#problem" class="headerlink" title="problem"></a>problem</h2><p>Programmers use a lot of symbols, often encoded with several characters. For the human brain, sequences like ->, <= or := are single logical tokens, even if they take two or three characters on the screen. Your eye spends a non-zero amount of energy to scan, parse and join multiple characters into a single logical one. Ideally, all programming languages should be designed with full-fledged Unicode symbols for operators, but that’s not the case yet.</p><p><img src="https://github.com/tonsky/FiraCode/raw/master/showcases/all_ligatures.png" alt="all_ligatures"></p><h2 id="installation"><a href="#installation" class="headerlink" title="installation"></a>installation</h2><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> brew tap caskroom/fonts</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> brew cask install font-fira-code</span></span><br></pre></td></tr></table></figure><h2 id="visual-studio-code"><a href="#visual-studio-code" class="headerlink" title="visual studio code"></a>visual studio code</h2><p>Add <code>"editor.fontLigatures": true</code> to <code>settings.json</code> :</p><p>To open <code>settings.json</code> , from the File menu choose Preferences, Settings or use keyboard shortcut <code>Cmd</code> + <code>,</code> . Then paste the following lines and save the file.</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">"editor.fontFamily"</span>: <span class="string">"Fira Code"</span>,</span><br><span class="line"><span class="string">"editor.fontLigatures"</span>: <span class="literal">true</span></span><br></pre></td></tr></table></figure><h2 id="what-it-looks-like"><a href="#what-it-looks-like" class="headerlink" title="what it looks like"></a>what it looks like</h2><h3 id="Go"><a href="#Go" class="headerlink" title="Go"></a>Go</h3><p><img src="https://github.com/tonsky/FiraCode/raw/master/showcases/go.png" alt="go"></p><h3 id="Swift"><a href="#Swift" class="headerlink" title="Swift"></a>Swift</h3><p><img src="https://github.com/tonsky/FiraCode/raw/master/showcases/swift.png" alt="swift"></p><h3 id="Ruby"><a href="#Ruby" class="headerlink" title="Ruby"></a>Ruby</h3><p><img src="https://github.com/tonsky/FiraCode/raw/master/showcases/ruby.png" alt="ruby"></p><h3 id="JavaScript"><a href="#JavaScript" class="headerlink" title="JavaScript"></a>JavaScript</h3><p><img src="https://github.com/tonsky/FiraCode/raw/master/showcases/javascript.png" alt="js"></p>]]></content>
<summary type="html">
<p>!!! From: <code>https://github.com/tonsky/FiraCode</code> !!!</p>
<h2 id="problem"><a href="#problem" class="headerlink" title="problem">
</summary>
</entry>
<entry>
<title>Calculate 24</title>
<link href="https://oceanoverflow.github.io/2018/06/12/Calculate24/"/>
<id>https://oceanoverflow.github.io/2018/06/12/Calculate24/</id>
<published>2018-06-12T06:53:49.000Z</published>
<updated>2018-06-26T08:44:01.357Z</updated>
<content type="html"><![CDATA[<p>算24点还是挺考验一个人的心算能力的，作为一个数学比较差劲的人，每次和别人玩24点总是会慢一个节拍，可以说是非常伤自尊了。不过虽然心算算不过别人，我们可以写一个自动计算24点的程序和别人比嘛，嘻嘻。</p><h2 id="expression-tree"><a href="#expression-tree" class="headerlink" title="expression tree"></a>expression tree</h2><p><img src="https://upload.wikimedia.org/wikipedia/commons/thumb/9/98/Exp-tree-ex-11.svg/500px-Exp-tree-ex-11.svg.png" alt="ExpressionTree"></p><p>我们可以利用表达式树来确定运算的优先级，进而确定运算的顺序，表达式树属于二叉树，其叶结点存储运算数（ <code>val</code> ），非叶子结点存储运算符 <code>op</code> （ <code>+，-，*，/</code> ）及运算结果（ <code>val = l.val op r.val</code> ）。每个节点都有指向其左子节点（ <code>l</code> ）和右子节点（ <code>r</code> ）的指针，叶节点中这两个指针均为空。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> node <span class="keyword">struct</span> {</span><br><span class="line"> l *node</span><br><span class="line"> r *node</span><br><span class="line"> op <span class="keyword">rune</span></span><br><span class="line"> val <span class="keyword">float64</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(n *node)</span> <span class="title">set</span><span class="params">(l, r *node, op <span class="keyword">rune</span>)</span></span> {</span><br><span class="line"> n.l = l</span><br><span class="line"> n.r = r</span><br><span class="line"> n.op = op</span><br><span class="line"> <span class="keyword">switch</span> op {</span><br><span class="line"> <span class="keyword">case</span> <span class="string">'+'</span>:</span><br><span class="line"> n.val = <span class="keyword">float64</span>(n.l.val + n.r.val)</span><br><span class="line"> <span class="keyword">case</span> <span class="string">'-'</span>:</span><br><span class="line"> n.val = <span class="keyword">float64</span>(n.l.val - n.r.val)</span><br><span class="line"> <span class="keyword">case</span> <span class="string">'*'</span>:</span><br><span class="line"> n.val = <span class="keyword">float64</span>(n.l.val * n.r.val)</span><br><span class="line"> <span class="keyword">case</span> <span class="string">'/'</span>:</span><br><span class="line"> n.val = <span class="keyword">float64</span>(n.l.val / n.r.val)</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(n *node)</span> <span class="title">String</span><span class="params">()</span> <span class="title">string</span></span> {</span><br><span class="line"> <span class="keyword">if</span> n.op != <span class="keyword">rune</span>(<span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">return</span> fmt.Sprintf(<span class="string">" %c "</span>, n.op)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> fmt.Sprintf(<span class="string">"%d"</span>, <span class="keyword">int</span>(n.val))</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="build-tree"><a href="#build-tree" class="headerlink" title="build tree"></a>build tree</h2><p>由于存在运算优先级的原因，所以在运算数和运算符都相同的情况下当其运算顺序不同时得出的表达式树的形态也不同，例如 <code>(1 + 8) * (7 - 13)</code> 与 <code>1 + 8 * 7 - 13</code> 的运算顺序是不同的，<code>(1 + 8) * (7 - 13)</code> 对应的是下面第一棵树的形态， <code>1 + 8 * 7 - 13</code> 对应的是下面第四棵树的形态。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"> x x x x x</span><br><span class="line"> / \ / \ / \ / \ / \</span><br><span class="line"> x x x x x x x x x x</span><br><span class="line"> / \ / \ / \ / \ / \ / \</span><br><span class="line">x x x x x x x x x x x x</span><br><span class="line"> / \ / \ / \ / \</span><br><span class="line"> x x x x x x x x</span><br></pre></td></tr></table></figure><p>为了得出所有的可能结果，我们需要构建所有不同形态的表达式树，<code>buildAllTrees</code> 利用递归方法来建树，例如给定 <code>3，4，5，6</code> 这四个数，可以构建出上面5种形态的树。 <code>3，4，5，6</code> 作为参数传入，然后将数组分割为两个部分，例如上面的 <code>3，4，5，6</code> 可以分割为 <code>3</code> 和 <code>4，5，6</code> 或者 <code>3，4</code> 和 <code>5，6</code> 或者 <code>3，4，5</code> 和 <code>6</code> ，然后对这两个部分分别再进行递归建树操作，当数组长度为1时也就是当 <code>len(s) == 1</code> 作为递归函数退出的条件，此时返回一个叶节点，按照这样的方式最终可以得到所有可能形态的树的根节点。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">buildAllTrees</span><span class="params">(s []<span class="keyword">int</span>)</span> []*<span class="title">node</span></span> {</span><br><span class="line"> <span class="keyword">if</span> <span class="built_in">len</span>(s) == <span class="number">1</span> {</span><br><span class="line"> tree := &node{result: <span class="keyword">float64</span>(s[<span class="number">0</span>])}</span><br><span class="line"> <span class="keyword">return</span> []*node{tree}</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> treelist := []*node{}</span><br><span class="line"> <span class="keyword">for</span> i := <span class="number">1</span>; i < <span class="built_in">len</span>(s); i++ {</span><br><span class="line"> left := s[:i]</span><br><span class="line"> right := s[i:]</span><br><span class="line"> leftTrees := buildAllTrees(left)</span><br><span class="line"> rightTrees := buildAllTrees(right)</span><br><span class="line"> <span class="keyword">for</span> _, l := <span class="keyword">range</span> leftTrees {</span><br><span class="line"> <span class="keyword">for</span> _, r := <span class="keyword">range</span> rightTrees {</span><br><span class="line"> combinedTree := build(l, r)</span><br><span class="line"> treelist = <span class="built_in">append</span>(treelist, combinedTree...)</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> treelist</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>因为存在四种运算符，例如利用 <code>a</code> 和 <code>b</code> 这两个节点构建树的时候，可以有 <code>a+b</code> ， <code>a-b</code> ， <code>a*b</code> ， <code>a/b</code> 这四种可能，所以要分别列举。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">build</span><span class="params">(l, r *node)</span> []*<span class="title">node</span></span> {</span><br><span class="line"> <span class="keyword">var</span> list []*node</span><br><span class="line"> tree1 := &node{}</span><br><span class="line"> tree1.set(l, r, <span class="string">'+'</span>)</span><br><span class="line"> list = <span class="built_in">append</span>(list, tree1)</span><br><span class="line"> tree2 := &node{}</span><br><span class="line"> tree2.set(l, r, <span class="string">'-'</span>)</span><br><span class="line"> list = <span class="built_in">append</span>(list, tree2)</span><br><span class="line"> tree3 := &node{}</span><br><span class="line"> tree3.set(l, r, <span class="string">'*'</span>)</span><br><span class="line"> list = <span class="built_in">append</span>(list, tree3)</span><br><span class="line"> <span class="keyword">if</span> r.result != <span class="number">0.0</span> {</span><br><span class="line"> tree4 := &node{}</span><br><span class="line"> tree4.set(l, r, <span class="string">'/'</span>)</span><br><span class="line"> list = <span class="built_in">append</span>(list, tree4)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> list</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>我们可以通过中序遍历表达式树来遍历输出构建的表达式，由于上面我们定义了每个节点的 <code>String</code> 方法，如果节点为运算符，则输出其 <code>op</code> 的值，如果节点为叶结点，则输出其 <code>val</code> 值。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">in</span><span class="params">(root *node)</span></span> {</span><br><span class="line"> <span class="keyword">if</span> n == <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> n.op != <span class="keyword">rune</span>(<span class="number">0</span>) {</span><br><span class="line"> fmt.Print(<span class="string">"("</span>)</span><br><span class="line"> }</span><br><span class="line"> in(root.l)</span><br><span class="line"> fmt.Print(n.String())</span><br><span class="line"> in(root.r)</span><br><span class="line"> <span class="keyword">if</span> n.op != <span class="keyword">rune</span>(<span class="number">0</span>) {</span><br><span class="line"> fmt.Print(<span class="string">")"</span>)</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="permutation"><a href="#permutation" class="headerlink" title="permutation"></a>permutation</h2><p>上面的方法是对一个数组其固定的顺序构建表达式树，例如 <code>3，4，5，6</code> 。为了获得所有的运算表达式，我们需要对数组的所有排列组合进行表达式树的构建，例如对 <code>6，5，4，3</code> ，<code>5，6，4，3</code> 这样的排列组合构建表达式树。也就是说除了树形态不同的这点区别之外，还要注意叶结点值存储顺序不同的带来的可能（可能会导致重复）。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">Perm</span><span class="params">(a []<span class="keyword">int</span>, f <span class="keyword">func</span>([]<span class="keyword">int</span>)</span>)</span> {</span><br><span class="line"> perm(a, f, <span class="number">0</span>)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">perm</span><span class="params">(a []<span class="keyword">int</span>, f <span class="keyword">func</span>([]<span class="keyword">int</span>)</span>, <span class="title">i</span> <span class="title">int</span>)</span> {</span><br><span class="line"> <span class="keyword">if</span> i > <span class="built_in">len</span>(a) {</span><br><span class="line"> f(a)</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"> }</span><br><span class="line"> perm(a, f, i+<span class="number">1</span>)</span><br><span class="line"> <span class="keyword">for</span> j := i + <span class="number">1</span>; j < <span class="built_in">len</span>(a); j++ {</span><br><span class="line"> a[i], a[j] = a[j], a[i]</span><br><span class="line"> perm(a, f, i+<span class="number">1</span>)</span><br><span class="line"> a[i], a[j] = a[j], a[i]</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="main"><a href="#main" class="headerlink" title="main"></a>main</h2><p>在主函数中我们每隔五秒钟获得一个随机的扑克牌数字，然后对于该组数字所有的排列组合分别构建表达式树，对每棵构建好的树判断其根节点的值是否接近于24（由于涉及到除法操作，所以有小数点），如果接近，则将其表达式通过中序遍历输出，这样就可以得出一组数字所有24点的可能了。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> <span class="keyword">for</span> {</span><br><span class="line"> s := rand24()</span><br><span class="line"> fmt.Println(s)</span><br><span class="line"> Perm(s, <span class="function"><span class="keyword">func</span><span class="params">(a []<span class="keyword">int</span>)</span></span> {</span><br><span class="line"> treelist := buildAllTrees(a)</span><br><span class="line"> <span class="keyword">for</span> _, root := <span class="keyword">range</span> treelist {</span><br><span class="line"> <span class="keyword">if</span> math.Abs(<span class="number">24</span>-tree.result) < <span class="number">0.000001</span> {</span><br><span class="line"> in(root)</span><br><span class="line"> fmt.Println()</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> })</span><br><span class="line"> time.Sleep(<span class="number">5</span> * time.Second)</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">rand24</span><span class="params">()</span> <span class="params">(r []<span class="keyword">int</span>)</span></span> {</span><br><span class="line"> rand.Seed(time.Now().UnixNano())</span><br><span class="line"></span><br><span class="line"> club := []<span class="keyword">int</span>{<span class="number">1</span>, <span class="number">2</span>, <span class="number">3</span>, <span class="number">4</span>, <span class="number">5</span>, <span class="number">6</span>, <span class="number">7</span>, <span class="number">8</span>, <span class="number">9</span>, <span class="number">10</span>, <span class="number">11</span>, <span class="number">12</span>, <span class="number">13</span>}</span><br><span class="line"> heart := []<span class="keyword">int</span>{<span class="number">1</span>, <span class="number">2</span>, <span class="number">3</span>, <span class="number">4</span>, <span class="number">5</span>, <span class="number">6</span>, <span class="number">7</span>, <span class="number">8</span>, <span class="number">9</span>, <span class="number">10</span>, <span class="number">11</span>, <span class="number">12</span>, <span class="number">13</span>}</span><br><span class="line"> spade := []<span class="keyword">int</span>{<span class="number">1</span>, <span class="number">2</span>, <span class="number">3</span>, <span class="number">4</span>, <span class="number">5</span>, <span class="number">6</span>, <span class="number">7</span>, <span class="number">8</span>, <span class="number">9</span>, <span class="number">10</span>, <span class="number">11</span>, <span class="number">12</span>, <span class="number">13</span>}</span><br><span class="line"> diamond := []<span class="keyword">int</span>{<span class="number">1</span>, <span class="number">2</span>, <span class="number">3</span>, <span class="number">4</span>, <span class="number">5</span>, <span class="number">6</span>, <span class="number">7</span>, <span class="number">8</span>, <span class="number">9</span>, <span class="number">10</span>, <span class="number">11</span>, <span class="number">12</span>, <span class="number">13</span>}</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">var</span> all []<span class="keyword">int</span></span><br><span class="line"> all = <span class="built_in">append</span>(all, spade...)</span><br><span class="line"> all = <span class="built_in">append</span>(all, heart...)</span><br><span class="line"> all = <span class="built_in">append</span>(all, diamond...)</span><br><span class="line"> all = <span class="built_in">append</span>(all, club...)</span><br><span class="line"></span><br><span class="line"> r = <span class="built_in">make</span>([]<span class="keyword">int</span>, <span class="number">4</span>)</span><br><span class="line"></span><br><span class="line"> exist = <span class="built_in">make</span>(<span class="keyword">map</span>[<span class="keyword">int</span>]<span class="keyword">bool</span>)</span><br><span class="line"> <span class="keyword">for</span> i := <span class="number">0</span>; i < <span class="number">4</span>; i++ {</span><br><span class="line"> <span class="keyword">for</span> {</span><br><span class="line"> x := rand.Int() % <span class="built_in">len</span>(all)</span><br><span class="line"> <span class="keyword">if</span> !exist[x] {</span><br><span class="line"> exist[x] = <span class="literal">true</span></span><br><span class="line"> r[i] = all[x]</span><br><span class="line"> <span class="keyword">break</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>算24点还是挺考验一个人的心算能力的，作为一个数学比较差劲的人，每次和别人玩24点总是会慢一个节拍，可以说是非常伤自尊了。不过虽然心算算不过别人，我们可以写一个自动计算24点的程序和别人比嘛，嘻嘻。</p>
<h2 id="expression-tree"><a href=
</summary>
</entry>
<entry>
<title>Chat Chat Chat</title>
<link href="https://oceanoverflow.github.io/2018/06/11/ChatChatChat/"/>
<id>https://oceanoverflow.github.io/2018/06/11/ChatChatChat/</id>
<published>2018-06-11T07:09:34.000Z</published>
<updated>2018-06-26T08:44:07.665Z</updated>
<content type="html"><![CDATA[<p>承认吧，你肯定也会有深夜寂寞时特别想找人聊天但翻遍朋友圈，却还是没有发现合适的人的时候，今天我们就来写一个随机配对聊天程序，解决广大单身狗寂寞时想要找人聊天的需求吧。</p><p>为了让聊天不那么 <code>gay</code> ，我们的聊天程序会让异性匹配聊天，用户通过客户端连接到聊天程序服务器 <code>localhost:8080</code> ，例如使用 <code>netcat</code> 通过 <code>nc localhost 8080</code> 命令连接聊天程序服务器，服务器会根据你的性别匹配异性聊天。</p><p>在实现方面，为了达到异性相互聊天的目标，我们创建两个通道，分别是 <code>boy</code> 通道和 <code>girl</code> 通道，并通过 <code>net.Listen</code> 方法监听 <code>8080</code> 端口，对于每个连接我们起一个 <code>goroutine</code> 进行匹配操作。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> boy = <span class="built_in">make</span>(<span class="keyword">chan</span> io.ReadWriteCloser)</span><br><span class="line"><span class="keyword">var</span> girl = <span class="built_in">make</span>(<span class="keyword">chan</span> io.ReadWriteCloser)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> l, err := net.Listen(<span class="string">"tcp"</span>, <span class="string">"localhost:8080"</span>)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="built_in">panic</span>(err)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> {</span><br><span class="line"> c, err := l.Accept()</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> log.Println(err)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">go</span> match(c)</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>match</code> 方法会首先询问你的性别，如果是男生则匹配女生聊天，反之亦然。如果当前只有一个男生连进来，程序会执行第一个 <code>select</code> 中的 <code>case boy <- c:</code> 条件，由于 <code>boy</code> 通道是 <code>unbuffered channel</code> ，直到有女生进来之前该操作都会阻塞。如果此时进来一个女生，程序会执行第二个<code>select</code> 操作，由于 <code>girl</code> 通道也是 <code>unbuffered channel</code> ，所以<code>case girl <- c:</code> 语句也会阻塞，但此时 <code>case p := <-boy:</code> 会被执行，这时候就可以通过 <code>chat</code> 方法让两个 <code>net.Conn</code> 相互交流了。<code>match</code> 方法实现的逻辑就达到了异性交流的目的了。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">match</span><span class="params">(c io.ReadWriteCloser)</span></span> {</span><br><span class="line"> fmt.Fprint(c, <span class="string">"Are you a boy or a girl? "</span>)</span><br><span class="line"> <span class="keyword">var</span> sex <span class="keyword">string</span></span><br><span class="line"> fmt.Fscanln(c, &sex)</span><br><span class="line"> <span class="keyword">if</span> strings.HasPrefix(<span class="string">"boy"</span>, sex) {</span><br><span class="line"> <span class="keyword">select</span> {</span><br><span class="line"> <span class="keyword">case</span> boy <- c: </span><br><span class="line"> <span class="keyword">case</span> p := <-girl:</span><br><span class="line"> chat(p, c)</span><br><span class="line"> } </span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> strings.HasPrefix(<span class="string">"girl"</span>, sex) {</span><br><span class="line"> <span class="keyword">select</span> {</span><br><span class="line"> <span class="keyword">case</span> girl <- c:</span><br><span class="line"> <span class="keyword">case</span> p := <-boy:</span><br><span class="line"> chat(p, c)</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>上面的 <code>chat</code> 方法依赖 <code>cp</code> 方法 ，该方法封装了 <code>io.Copy</code> 方法，主要就是为了当任意一方结束聊天时发送一个 <code>err</code> 到我们的 <code>errc</code> ，以关闭双方的连接。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">cp</span><span class="params">(w io.Writer, r io.Reader, errc <span class="keyword">chan</span><- error)</span></span> {</span><br><span class="line"> _, err := io.Copy(w, r)</span><br><span class="line"> errc <- err</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>chat</code> 方法通过两个 <code>goroutine</code> ，<code>go cp(a, b, errc)</code> 和 <code>go cp(b, a, errc)</code> 使a和b这两个 <code>net.Conn</code> 相互发送消息，如果任意一方结束聊天， <code>errc</code> 通道就会返回一个 <code>error</code> 值，a和b就会被关闭，此时聊天结束。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">chat</span><span class="params">(a, b io.ReadWriteCloser)</span></span> {</span><br><span class="line"> fmt.Fprintln(a, <span class="string">"Found one! say hi"</span>)</span><br><span class="line"> fmt.Fprintln(b, <span class="string">"Found one! say hi"</span>)</span><br><span class="line"> errc := <span class="built_in">make</span>(<span class="keyword">chan</span> error, <span class="number">1</span>)</span><br><span class="line"> <span class="keyword">go</span> cp(a, b, errc) </span><br><span class="line"> <span class="keyword">go</span> cp(b, a, errc)</span><br><span class="line"> <span class="keyword">if</span> err := <- errc; err != <span class="literal">nil</span> {</span><br><span class="line"> log.Println(err)</span><br><span class="line"> }</span><br><span class="line"> a.Close()</span><br><span class="line"> b.Close()</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>承认吧，你肯定也会有深夜寂寞时特别想找人聊天但翻遍朋友圈，却还是没有发现合适的人的时候，今天我们就来写一个随机配对聊天程序，解决广大单身狗寂寞时想要找人聊天的需求吧。</p>
<p>为了让聊天不那么 <code>gay</code> ，我们的聊天程序会让异性匹配聊天，用户通
</summary>
</entry>
<entry>
<title>bitcoin</title>
<link href="https://oceanoverflow.github.io/2018/06/06/bitcoin/"/>
<id>https://oceanoverflow.github.io/2018/06/06/bitcoin/</id>
<published>2018-06-06T14:19:02.000Z</published>
<updated>2018-06-26T08:43:51.798Z</updated>
<content type="html"><![CDATA[<p>如果有时光机的话，我要做的第一件事情肯定就是穿梭回过去买比特币：），算了，不意淫了，还是学习要紧，虽然再也不能靠买比特币一夜暴富了，但是自己按照比特币实现一个山寨比特币的能力还是有的，比特币可以说是现在大红大紫的区块链的始祖，要全部了解它的原理可能还是要埋头啃个几个月的（并不简单）。</p><p>虽然比特币交易确认时间长，消耗资源多，但依旧无法掩盖其作为区块链始祖的光环。</p><p><img src="https://cdn-images-1.medium.com/max/1600/1*_RY4v9D5-BOVMzFMr8Dk4A.png" alt="bitcoin"></p><h2 id="Block-amp-BlockChain"><a href="#Block-amp-BlockChain" class="headerlink" title="Block & BlockChain"></a>Block & BlockChain</h2><p>比特币中有两个重要的基本概念，区块和区块链，如果将区块想象成珍珠，那么区块链就是将这些珍珠都串起来的珍珠项链，这个珍珠项链没有尽头，新加入的珍珠和次新的珍珠彼此靠近，并通过某种关系紧密联系在一起，牢不可破。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> Block <span class="keyword">struct</span> {</span><br><span class="line"> idx <span class="keyword">int</span></span><br><span class="line"> timestamp <span class="keyword">string</span></span><br><span class="line"> hash <span class="keyword">string</span></span><br><span class="line"> prevHash <span class="keyword">string</span></span><br><span class="line"> difficulty <span class="keyword">int</span></span><br><span class="line"> nonce <span class="keyword">string</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> BlockChain <span class="keyword">struct</span> {</span><br><span class="line"> sync.Mutex</span><br><span class="line"> blocks []Block</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(bc *BlockChain)</span> <span class="title">AddBlock</span><span class="params">(block Block)</span></span> {</span><br><span class="line"> bc.Lock()</span><br><span class="line"> <span class="keyword">defer</span> bc.Unlock()</span><br><span class="line"> bc.blocks = <span class="built_in">append</span>(bc.blocks, block)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">var</span> blockchain BlockChain</span><br></pre></td></tr></table></figure><h2 id="Proof-of-Work"><a href="#Proof-of-Work" class="headerlink" title="Proof of Work"></a>Proof of Work</h2><p>生成区块的过程就相当于打磨珍珠的过程，只有符合要求的珍珠才能最终被加入到珍珠项链中去，那么话说回来生成区块的过程是怎么样的呢，对于一个特定的区块，需要不断改变其中一个值（nonce），这里的 <code>nonce</code> 值从0开始逐渐递增，然后计算特定 <code>nonce</code> 对应的区块哈希值，只有哈希值符合相应的要求之后，才能说这个区块是有效的，才能被加入到区块链中，生成区块的过程也被叫做挖矿。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">generateBlock</span><span class="params">(oldBlock Block, difficulty <span class="keyword">int</span>)</span> <span class="title">Block</span></span> {</span><br><span class="line"> <span class="keyword">var</span> newBlock Block</span><br><span class="line"></span><br><span class="line"> newBlock.idx = oldBlock.idx + <span class="number">1</span></span><br><span class="line"> newBlock.timestamp = time.Now().String()</span><br><span class="line"> newBlock.prevHash = oldBlock.hash</span><br><span class="line"> newBlock.difficulty = diff</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> i := <span class="number">0</span>; ; i++ {</span><br><span class="line"> hex := fmt.Sprintf(<span class="string">"%x"</span>, i)</span><br><span class="line"> newBlock.nonce = hex</span><br><span class="line"> <span class="keyword">if</span> !isHashValid(calculateHash(newBlock), newBlock.difficulty) {</span><br><span class="line"> <span class="keyword">continue</span></span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> fmt.Println(calculateHash(newBlock), <span class="string">" work done!"</span>)</span><br><span class="line"> newBlock.Hash = calculateHash(newBlock)</span><br><span class="line"> <span class="keyword">break</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> newBlock</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">calculateHash</span><span class="params">(block Block)</span> <span class="title">string</span></span> {</span><br><span class="line"> record := strconv.Itoa(block.idx) + block.timestamp + block.prevHash + block.difficulty + block.nonce</span><br><span class="line"> h := sha256.New()</span><br><span class="line"> h.Write([]<span class="keyword">byte</span>(record))</span><br><span class="line"> hashed := h.Sum(<span class="literal">nil</span>)</span><br><span class="line"> <span class="keyword">return</span> hex.EncodeToString(hashed)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>那么什么样的区块才是有效的呢，简单来说就是对于一个哈希值，其开头部分都是0就可以了（0越多，难度越高）。像这种，找到答案很困难但验证答案却非常轻松的算法就是 <code>Proof of Work</code> 的精髓。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">isHashValid</span><span class="params">(hash <span class="keyword">string</span>, difficulty <span class="keyword">int</span>)</span> <span class="title">bool</span></span> {</span><br><span class="line"> prefix := strings.Repeat(<span class="string">"0"</span>, difficulty)</span><br><span class="line"> <span class="keyword">return</span> strings.HasPrefix(hash, prefix)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="一个难度逐渐递增的区块链"><a href="#一个难度逐渐递增的区块链" class="headerlink" title="一个难度逐渐递增的区块链"></a>一个难度逐渐递增的区块链</h2><p>下面实现了一个难度逐渐递增的区块链，运行时就会发现产生区块的时间间隔越来越长，因为 <code>CPU</code> 需要经过更多的计算才能找到符合条件的哈希值。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> difficulty := <span class="number">1</span></span><br><span class="line"> genesisBlock := Block{}</span><br><span class="line"> genesisBlock = Block{</span><br><span class="line"> idx: <span class="number">0</span>, </span><br><span class="line"> timestamp: time.Now().String(), </span><br><span class="line"> hash: calculateHash(genesisBlock),</span><br><span class="line"> prevHash: <span class="string">""</span>, </span><br><span class="line"> difficulty: difficulty, </span><br><span class="line"> nonce: <span class="string">""</span>,</span><br><span class="line"> }</span><br><span class="line"> blockchain.AddBlock(genesisBlock)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">var</span> block Block</span><br><span class="line"> block = genesisBlock</span><br><span class="line"> <span class="keyword">for</span> {</span><br><span class="line"> block = generateBlock(block, diff)</span><br><span class="line"> difficulty++</span><br><span class="line"> blockchain.AddBlock(block)</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>如果有时光机的话，我要做的第一件事情肯定就是穿梭回过去买比特币：），算了，不意淫了，还是学习要紧，虽然再也不能靠买比特币一夜暴富了，但是自己按照比特币实现一个山寨比特币的能力还是有的，比特币可以说是现在大红大紫的区块链的始祖，要全部了解它的原理可能还是要埋头啃个几个月的（并
</summary>
</entry>
<entry>
<title>Magic Number</title>
<link href="https://oceanoverflow.github.io/2018/06/05/MagicNumber/"/>
<id>https://oceanoverflow.github.io/2018/06/05/MagicNumber/</id>
<published>2018-06-05T08:27:07.000Z</published>
<updated>2018-06-26T08:46:45.861Z</updated>
<content type="html"><![CDATA[<p>微笑.jpg，这里年轻人们聊天时经常使用的说法，但是！作为一个严谨的程序员，我们能不能通过文件的后缀来判断这个文件的类型呢？例如看到电脑上存在“微笑.jpg”这个文件，通过后缀 <code>jpg</code> 来判断它就是一个 <code>JPEG</code> 文件呢，答案肯定是否定的，因为文件名可以被轻松篡改，例如这个文件本来是“微笑.gif”，有人想不开就将它改成“微笑.jpg”，虽然没有什么影响，但是可以说明一个问题，我们不能通过文件后缀来判断文件的类型。</p><p>如果不使用文件后缀，我们应该如何判断文件的类型呢，答案就是我们今天要讨论的<code>Magic Number</code> ，即魔术数，因为文件的本质就是字节数组，所以在一般的操作系统中，对于特定类型的文件，其开头的几个字节都是相同的，例如对于 <code>JPEG</code> 格式的文件，开头的字节都是 <code>\xff\xd8\xff</code> ，我们可以通过读取文件开头的字节来确定文件的类型，这种方法准确性更高。</p><p>下面我们准备一个魔术数到类型的字典，注意，这里只列举了几个例子，还可以按照这样的格式增加字典项。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> magicNumber = <span class="keyword">map</span>[<span class="keyword">string</span>]<span class="keyword">string</span> {</span><br><span class="line"> <span class="string">"\xff\xd8\xff"</span>: <span class="string">"image/jpeg"</span>,</span><br><span class="line"> <span class="string">"\x89PNG\r\n\x1a\n"</span>: <span class="string">"image/png"</span>,</span><br><span class="line"> <span class="string">"GIF87a"</span>: <span class="string">"image/gif"</span>,</span><br><span class="line"> <span class="string">"GIF89a"</span>: <span class="string">"image/gif"</span>,</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">getFileType</span><span class="params">(incipit []<span class="keyword">byte</span>)</span> <span class="title">string</span></span> {</span><br><span class="line"> incipitStr := <span class="keyword">string</span>(incipit)</span><br><span class="line"> <span class="keyword">for</span> magic, mime := <span class="keyword">range</span> magicNumber {</span><br><span class="line"> <span class="keyword">if</span> strings.HasPrefix(incipitStr, magic) {</span><br><span class="line"> <span class="keyword">return</span> mime</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="string">""</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>真正匹配格式的逻辑比较简单，遍历 <code>magicNumber</code> 这个 <code>map</code> ，判断每个 <code>key</code> 是不是某个字符串的前缀，因为魔术数符合前缀编码的特点，即任何一个编码都不是另一个编码的前缀，所以我们能保证判断文件类型时，不会有二义。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> filepath := os.Args[<span class="number">1</span>]</span><br><span class="line"> file, err := os.Open(filepath)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="built_in">panic</span>(err)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">defer</span> file.Close()</span><br><span class="line"></span><br><span class="line"> buff := <span class="built_in">make</span>([]<span class="keyword">byte</span>, <span class="number">512</span>)</span><br><span class="line"> _, err = file.Read(buff)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="built_in">panic</span>(err)</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> file.Seek(<span class="number">0</span>, os.SEEK_SET)</span><br><span class="line"> typ := getFileType(buff)</span><br><span class="line"> fmt.Println(typ)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="gate-upload"><a href="#gate-upload" class="headerlink" title="gate upload"></a>gate upload</h2><p>下面我们来看一个 <code>Magic Number</code> 的应用场景，例如我们有一个图片上传程序，但是只能允许上传 <code>JPEG</code> 格式的文件，给定一个 <code>io.Reader</code> ， 我们先读取前两个字节，判断其值是否等于 <code>JPEG</code> 的 <code>Magic Number</code> ，如果不等，则直接停止上传，如果相等，则继续上传。由于此时 <code>r</code> 已经读取了两个字节了，所以需要重新构造一个 <code>io.Reader</code>（ <code>r := io.MultiReader(bytes.NewReader(b), r)</code> ）继续上传。利用 <code>Magic Number</code> 我们可以避免将整个文件读入内存中，增加程序的运行效率。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">handleUpload</span><span class="params">(r io.Reader)</span> <span class="title">error</span></span> {</span><br><span class="line"> b := <span class="built_in">make</span>([]<span class="keyword">byte</span>, <span class="number">2</span>)</span><br><span class="line"> _, err := r.Read(b)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> err</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> jpeg := []<span class="keyword">byte</span>{<span class="number">0xFF</span>, <span class="number">0xD8</span>}</span><br><span class="line"> <span class="keyword">if</span> bytes.Equal(b, jprg) {</span><br><span class="line"> <span class="keyword">return</span> error.New(<span class="string">"not a JPEG"</span>)</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> r := io.MultiReader(bytes.NewReader(b), r)</span><br><span class="line"></span><br><span class="line"> err = uploadFile(r)</span><br><span class="line"> <span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line"> <span class="keyword">return</span> err</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>微笑.jpg，这里年轻人们聊天时经常使用的说法，但是！作为一个严谨的程序员，我们能不能通过文件的后缀来判断这个文件的类型呢？例如看到电脑上存在“微笑.jpg”这个文件，通过后缀 <code>jpg</code> 来判断它就是一个 <code>JPEG</code> 文件呢，
</summary>
</entry>
<entry>
<title>Raft</title>
<link href="https://oceanoverflow.github.io/2018/06/03/Raft/"/>
<id>https://oceanoverflow.github.io/2018/06/03/Raft/</id>
<published>2018-06-03T12:59:23.000Z</published>
<updated>2018-06-26T08:48:15.739Z</updated>
<content type="html"><![CDATA[<ul><li>Raft stands for Replicated and Fault Tolerant</li><li>Raft is something you can build out of a collection of logs</li><li>Raft is something you can use to get away from the island of Paxos</li></ul><h2 id="Goal-Replicated-Log"><a href="#Goal-Replicated-Log" class="headerlink" title="Goal: Replicated Log"></a>Goal: Replicated Log</h2><ul><li>Replicated log => replicated state machine<ul><li>All servers execute same commands in same order</li></ul></li><li>Consensus module ensures proper log replication</li><li>System makes progress as long as any majority of servers are up</li><li>Failure model: fail-stop(not Byzantine), delayed/lost messages </li></ul><h2 id="Approaches-to-Consensus"><a href="#Approaches-to-Consensus" class="headerlink" title="Approaches to Consensus"></a>Approaches to Consensus</h2><p>Two possible approaches to consensus:</p><ul><li>Symmetric, leader-less:<ul><li>All servers have equal roles</li><li>Clients can contact any server </li></ul></li><li>Asymmetric, leader-based:<ul><li>At any given time, one server is in charge, others accept its decision</li><li>Clients communicate with the leader </li></ul></li><li>Raft uses a leader:<ul><li>Decomposes the problem (normal operation, leader changes)</li><li>Simplifies normal operation (no conflicts)</li><li>More efficient than leader-less approaches </li></ul></li></ul><h2 id="Leader-Election"><a href="#Leader-Election" class="headerlink" title="Leader Election"></a>Leader Election</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"> +------------------+</span><br><span class="line"> | Become Candidate |</span><br><span class="line"> +------------------+</span><br><span class="line"> |</span><br><span class="line"> v</span><br><span class="line"> +---------------+ timeout</span><br><span class="line"> | currentTerm++ |<--------------+</span><br><span class="line"> | vote for self | |</span><br><span class="line"> +---------------+ |</span><br><span class="line"> | |</span><br><span class="line"> v |</span><br><span class="line"> +-----------------------+ |</span><br><span class="line"> | Send RequestVote RPCs |-----------+</span><br><span class="line"> | to other servers |</span><br><span class="line"> +-----------------------+</span><br><span class="line">votes from majority | | RPC from Leader</span><br><span class="line"> v v</span><br><span class="line"> +----------------+ +---------+</span><br><span class="line"> | Become Leader, | | Become |</span><br><span class="line"> | send heartbeats| | Follower|</span><br><span class="line"> +----------------+ +---------+</span><br></pre></td></tr></table></figure><ul><li>Safety: allow at most one winner per term<ul><li>Each server gives only one vote per term(persist on disk)</li><li>Majority required to win election</li></ul></li></ul><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"> B can't also Voted for</span><br><span class="line"> get majority candidate A</span><br><span class="line">+-----------------+ +-------------------------+</span><br><span class="line">| +-----+ +-----+ | | +-----+ +-----+ +-----+ | </span><br><span class="line">| | | | | | | | | | | | | | </span><br><span class="line">| +-----+ +-----+ | | +-----+ +-----+ +-----+ |</span><br><span class="line">+-----------------+ +-------------------------+</span><br><span class="line"> Servers</span><br></pre></td></tr></table></figure><ul><li>Livebess: some candidate must eventually win<ul><li>Choose election timeout randomly in [T, 2T] (e.g. 150-300ms)</li><li>One server usually times out and wins election before others time out</li><li>Works well if T >> broadcast time</li></ul></li></ul><p>Randomized approach simpler than ranking</p><h2 id="Normal-Operation"><a href="#Normal-Operation" class="headerlink" title="Normal Operation"></a>Normal Operation</h2><ul><li>Client sends command to leader</li><li>Leader appends command to its log</li><li>Leader send AppendEntries RPCs to all followers</li><li>Once new entry committed<ul><li>Leader executes command in its state machine, returns result to client</li><li>Leader notifies followers of committed entries in subsequent AppendEntries RPCs</li><li>Followers execute committed commands in their state machines </li></ul></li><li>Crashed/slow followers<ul><li>Leader retries AppendEntries RPCs until they succeed</li></ul></li><li>Optimal performance in common case:<ul><li>One successful RPC to any majority of servers</li></ul></li></ul><h2 id="Log-Structure"><a href="#Log-Structure" class="headerlink" title="Log Structure"></a>Log Structure</h2><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> Command <span class="keyword">string</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> Log <span class="keyword">struct</span> {</span><br><span class="line"> Term <span class="keyword">int</span></span><br><span class="line"> Command Command</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> Logs []Log</span><br></pre></td></tr></table></figure><ul><li>Must survive crashes (store on disk)</li><li>Entry committed if safe to execute in state machines<ul><li>Replicated on majority of servers by leader of its term</li></ul></li></ul><h2 id="Log-Inconsistencies"><a href="#Log-Inconsistencies" class="headerlink" title="Log Inconsistencies"></a>Log Inconsistencies</h2><p>Crashes can result in log inconsistencies</p><ul><li>Raft minimizes special code for repairing inconsistencies<ul><li>Leader assumes its log is correct</li><li>Normal operation will repair all inconsistencies </li></ul></li></ul>]]></content>
<summary type="html">
<ul>
<li>Raft stands for Replicated and Fault Tolerant</li>
<li>Raft is something you can build out of a collection of logs</li>
<li>Raft is
</summary>
</entry>
<entry>
<title>Replicated Service Client</title>
<link href="https://oceanoverflow.github.io/2018/06/02/ReplicatedServiceClient/"/>
<id>https://oceanoverflow.github.io/2018/06/02/ReplicatedServiceClient/</id>
<published>2018-06-02T07:01:00.000Z</published>
<updated>2018-06-26T08:48:13.702Z</updated>
<content type="html"><![CDATA[<p>Self-explanatory :-)<br>Actually cause I feel lazy today.</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> ReplicatedClient <span class="keyword">interface</span> {</span><br><span class="line"> <span class="comment">// Init initializes the client to use the given servers.</span></span><br><span class="line"> <span class="comment">// To make a particular request later,</span></span><br><span class="line"> <span class="comment">// the client can use callOne(srv, args), where srv </span></span><br><span class="line"> <span class="comment">// is one of the servers from the list</span></span><br><span class="line"> Init(servers []<span class="keyword">string</span>, callOne(<span class="keyword">string</span>, Args) Reply)</span><br><span class="line"> </span><br><span class="line"> <span class="comment">// Call makes a request to an available server.</span></span><br><span class="line"> <span class="comment">// Multiple gouroutines may call Call concurrently.</span></span><br><span class="line"> Call(args Args) Reply </span><br><span class="line">}</span><br></pre></td></tr></table></figure><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> Client <span class="keyword">struct</span> {</span><br><span class="line"> servers []<span class="keyword">string</span></span><br><span class="line"> callOne <span class="function"><span class="keyword">func</span><span class="params">(<span class="keyword">string</span>, Args)</span> <span class="title">Reply</span></span></span><br><span class="line"><span class="function"></span></span><br><span class="line"><span class="function"> <span class="title">mu</span> <span class="title">sync</span>.<span class="title">Mutex</span></span></span><br><span class="line"><span class="function"> <span class="title">prefer</span> <span class="title">int</span></span></span><br><span class="line"><span class="function">}</span></span><br><span class="line"><span class="function"></span></span><br><span class="line"><span class="function"><span class="title">func</span> <span class="params">(c *Client)</span> <span class="title">Init</span><span class="params">(servers []<span class="keyword">string</span>, callOne <span class="keyword">func</span>(<span class="keyword">string</span>, Args)</span> <span class="title">Reply</span>)</span> {</span><br><span class="line"> c.servers = servers</span><br><span class="line"> c.callOne = callOne</span><br><span class="line">}</span><br></pre></td></tr></table></figure><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(c *Client)</span> <span class="title">Call</span><span class="params">(args Args)</span> <span class="title">Reply</span></span> {</span><br><span class="line"> <span class="keyword">type</span> result <span class="keyword">struct</span> {</span><br><span class="line"> serverID <span class="keyword">int</span></span><br><span class="line"> reply Reply</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> timeout = <span class="number">1</span> * time.Second</span><br><span class="line"> t := time.NewTimer(timeout)</span><br><span class="line"> <span class="keyword">defer</span> t.Stop()</span><br><span class="line"></span><br><span class="line"> done := <span class="built_in">make</span>(<span class="keyword">chan</span> result, <span class="number">1</span>)</span><br><span class="line"></span><br><span class="line"> c.mu.Lock()</span><br><span class="line"> prefer := c.prefer</span><br><span class="line"> c.mu.Unlock()</span><br><span class="line"></span><br><span class="line"> <span class="keyword">var</span> r result</span><br><span class="line"> <span class="keyword">for</span> off := <span class="number">0</span>; off < <span class="built_in">len</span>(c.servers); off++ {</span><br><span class="line"> id := (prefer + off) % <span class="built_in">len</span>(c.servers)</span><br><span class="line"> <span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line"> done <- result{id, c.callOne(c.servers[id], args)}</span><br><span class="line"> }()</span><br><span class="line"> <span class="keyword">select</span> {</span><br><span class="line"> <span class="keyword">case</span> r = <-done:</span><br><span class="line"> goro Done</span><br><span class="line"> <span class="keyword">case</span> <-t.C:</span><br><span class="line"> t.Reset(timeout)</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">Done:</span><br><span class="line"> c.mu.Lock()</span><br><span class="line"> c.prefer = r.serverID</span><br><span class="line"> c.mu.Unlock()</span><br><span class="line"> <span class="keyword">return</span> r.reply </span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>Self-explanatory :-)<br>Actually cause I feel lazy today.</p>
<figure class="highlight golang"><table><tr><td class="gutter"><pre><span c
</summary>
</entry>
<entry>
<title>Semaphore</title>
<link href="https://oceanoverflow.github.io/2018/06/01/Semaphore/"/>
<id>https://oceanoverflow.github.io/2018/06/01/Semaphore/</id>
<published>2018-06-01T13:07:44.000Z</published>
<updated>2018-06-26T08:48:01.794Z</updated>
<content type="html"><![CDATA[<p>现在要完成一个非常简单的任务，给定很多图片的 <code>URL</code> ，将它们下载到本地，你会怎么做？</p><h2 id="Sequential"><a href="#Sequential" class="headerlink" title="Sequential"></a>Sequential</h2><p>如果你想也不想，就写下了下面这种代码，说明你是一个能干活的程序员，但这并不意味着你是一个优秀的程序员。下面的代码虽然没有什么致命的错误，但是效率非常低，程序运行时 <code>CPU</code> 大部分时间都在等待网络 <code>I/O</code> 而处于空闲状态，导致 <code>CPU</code> 的利用率非常低。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">for</span> idx, task := <span class="keyword">range</span> tasks {</span><br><span class="line"> <span class="keyword">if</span> err := Do(idx, task); err != <span class="literal">nil</span> {</span><br><span class="line"> fmt.Println(err)</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="Parallel"><a href="#Parallel" class="headerlink" title="Parallel"></a>Parallel</h2><p>这种完全由相互独立的子任务组成的任务，被称为 <em>embarrassingly parallel</em> ，这种问题可以通过并行的方法提高程序的性能，而且并行程度越高，性能越好，在程序中我们通过添加 <code>go</code> 这个关键字来创建多个 <code>goroutine</code> ，使多个子任务并发执行，以减少网络 <code>I/O</code> 延迟，提高 <code>CPU</code> 的利用率 。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> wg sync.WaitGroup</span><br><span class="line"><span class="keyword">for</span> idx, task := <span class="keyword">range</span> tasks {</span><br><span class="line"> wg.Add(<span class="number">1</span>)</span><br><span class="line"> <span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">(i <span class="keyword">int</span>, v <span class="keyword">string</span>)</span></span> {</span><br><span class="line"> <span class="keyword">defer</span> wg.Done()</span><br><span class="line"> <span class="keyword">if</span> err := Do(i, v); err != <span class="literal">nil</span> {</span><br><span class="line"> fmt.Println(err)</span><br><span class="line"> }</span><br><span class="line"> }(idx, task)</span><br><span class="line">}</span><br><span class="line">wg.Wait()</span><br></pre></td></tr></table></figure><h2 id="Parallel-with-Semaphore"><a href="#Parallel-with-Semaphore" class="headerlink" title="Parallel with Semaphore"></a>Parallel with Semaphore</h2><p>但是通过创建多个 <code>goroutine</code> 提高并行度从而提高性能的方法有其局限性，例如一次性创建过多的网络连接，超出了进程打开文件的数量限制时，程序就会报错。所以我们需要限制单位时间内并行程序的个数，通过信号量（ <code>semaphore</code> ）来限制并行度。在 <code>golang</code> 中，我们可以使用容量为 n 的 <code>buffered channel</code> 来模拟 <code>counting semaphore</code> ，这样以来在每个子任务执行前需要通过 <code>sem <- struct{}{}</code> 来获取执行许可，如果 <code>sem</code> 通道已满，说明当前已经有n个子任务正在执行，该操作就会阻塞等待，直到通道空闲。当任务执行完成后，使用 <code><-sem</code> 释放执行许可。</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">sem := <span class="built_in">make</span>(<span class="keyword">chan</span> <span class="keyword">struct</span>{}, <span class="number">8</span>) <span class="comment">// 8 jobs at once</span></span><br><span class="line"><span class="keyword">var</span> wg sync.WaitGroup</span><br><span class="line"><span class="keyword">for</span> idx, task := <span class="keyword">range</span> tasks {</span><br><span class="line"> wg.Add(<span class="number">1</span>)</span><br><span class="line"> <span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">(i <span class="keyword">int</span>, v <span class="keyword">string</span>)</span></span> {</span><br><span class="line"> <span class="keyword">defer</span> wg.Done()</span><br><span class="line"> sem <- <span class="keyword">struct</span>{}{}</span><br><span class="line"> <span class="keyword">if</span> err := Do(i, v); err != <span class="literal">nil</span> {</span><br><span class="line"> fmt.Println(err)</span><br><span class="line"> }</span><br><span class="line"> <-sem</span><br><span class="line"> }(idx, task)</span><br><span class="line">}</span><br><span class="line">wg.Wait()</span><br><span class="line"><span class="built_in">close</span>(sem)</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>现在要完成一个非常简单的任务，给定很多图片的 <code>URL</code> ，将它们下载到本地，你会怎么做？</p>
<h2 id="Sequential"><a href="#Sequential" class="headerlink" title="Sequenti
</summary>
</entry>
</feed>