This is used to record the thoughts and results from the Drawabox exercise. At the same time, according to the 50% rule, I will be drawing figures or objects from the Atmosphere’s Sketch book as the random drawing. Maybe in the future, I will choose images from Pixiv.

Record of the house renting.

Notes about the drawabox.

• recommandations: https://drawabox.com/recommendations

随着在美国生活得越来越久，对车的需求也越来越大，比如带小狗去上training课程、去Costco买生活用品、旅游、与朋友聚会等等。之所以在美国呆了两年多才开始真正开始这件事，是因为一方面自己还是新手司机（开车上路总担心会不小心刮蹭，亦或者自己对道路状况的不熟悉可能会导致的安全问题），另一方面是自己不喜欢开车（租车行也好，Zipcar租车也好，总是开起来不舒服）。目前我开过的车有：

• Dodge Charger
• Kia Soul
• Honda HR-V
• Subaru XV Crosstrek
• Honda Civic
• Toyota Corolla
• Mitsubishi Outlander
• Mazda CX-5

我其实挺喜欢开轿车的，因为重心低并且方向操控感更好，但是空间太小，所以忽略掉了这个选择。SUV我之前是一直不喜欢的，直到有次偶然在Zipcar上租到了Mazda CX-5。驾驶体验特别好，这可能就是一见钟情吧哈哈哈。

现在是2022/01/05，在去年年末的时候汽车芯片突然短缺再加上长期的疫情影响，导致车价突增，但是我不想再拖这一件事了，而且看了一下history pricing，涨幅在5000刀左右，在可以接受的范围。没错，本人是心甘情愿的韭菜。

这篇笔记将会记录买车的全过程。

The Manager’s Path: A Guide for Tech Leaders Navigating Growth and Change is one of the books that I was referenced to prepare for the behavioral questions in the tech interviews. In general, there are four kinds of interview rounds: coding, system design, OOD, behavioral question. The BQ round is the only one associated with a manager, who has a lot of weight on the final decision to the offer.

Behavioral interview questions are those that focus on how you’ve handled various situations in the workplace and reveal your character traits, abilities and skills. These questions give an interviewer an idea of how you would behave if a similar situation were to arise, the logic being that your success in the past will show success in the future.

Unlike traditional interview questions, behavior job interview techniques look for concrete examples of skills and experiences that relate to the position. Your answers to these questions should provide a brief story that illustrates your skills and strengths as an employee. For each answer, give the interviewer the background to the story briefly, specific actions you took and the results.

This post includes the notes that I went through during the reading of the book as mentioned in title. I would like to paste one sentence that totally solved one of my questions regarding the differences between refactoring and design patterns:

When my coauthors and I wrote Design Patterns, we mentioned that design patterns provide targets for refactorings. However, identifying the target is only one part of the problem; transforming your code so that you get there is another challenge.

Project Information

1. Project Name

Dragonfly - an intelligent P2P based image and file distribution system.

2. Project Task Description

Now the client of Dragonfly will random read and write disk multiple times during the downloading process.
For directly using dfget to download a file:

• dfget random writes a piece into disk after downloading it
• dfget server random reads the piece from disk to share it
• dfget sequential reads the file from disk after downloading to do checksum

And for using dfdaemon to pull images, there’re extra disk IO by Dragonfly:

• dfdaemon sequential reads the file from disk to send it to dockerd

It’s not a problem when the host has a local disk. But it will be a potential
bottleneck when the Dragonfly client runs on a virtual machine with a cloud disk, all the disk IO will become network IO which has a bad performance when read/write at the same time.

So a solution is needed to reduce the IO times generated by Dragonfly.

3. Implementation Plan

As proposed in the Issue #1164, the P2P streaming is come up for the problem discussed above. And my work in ASoC 2020 is to implement part of the streaming download method. The task can be summaried into four aspects, which are supernode scheduler, dfget downloader, dfget uploader, IPC between user and Dragonfly stream dfget.

3.1 Supernode Scheduler

The work in supernode can be categorized into two respectives:

.png)

• Maintain the sliding window in supernode.

  • Initialize: The size of window will be registered in the dfget task registration phrase. And then, the window state is recorded in the ProgressManager as the SyncMap indexed by ClientID. The size of window is staic once it is recorded by supernode.
  • Update: The dfget client will report to the supernode about the result of downloading for every piece. In the report API of supernode server, the window will be updated based on the received piece number and piece status. To be specific, the window keeps sliding until the unacknowledged piece. It should be noted that the supernode is using the sender window, which has the range of [una, una + wnd).
  • Finish: Update the DeleteCID method of ProgressManager to delete the sliding window state if it is in stream mode.

• Schedule the pieces according to the window state

  The only modification that I made in this part lies at the GetPieceProgressByCID method of ProgressManager. The available pieces in regular mode means the success pieces which are not running; while for the stream mode, the available pieces means the cached pieces which are not running. After the modification, the ProgressManager would only return the available pieces inside the window when stream mode is on.

  A new kind of piece status is created: UNCACHED. When the piece is downloaded successfully, I assume that it will sotred into the cache immediately. Afterwards, the piece maybe popped out of the cache. And in this case, the handler of supernode server deletePieceCache at supernode/server/router.go:101 would be called to change the state of the piece.

  It should be noted that, the scheduler in stream mode currently uses the same scheduler as the regular mode, which may demands future optimization.

3.2 DFGET Downloader

Since the client stream writer has been implemented under the p2p_downloader folder, which means that the pieces downloading process has been finished. My work here is to fullfill the task of handling the successfully downloaded pieces to uploader. Besides that, in the perparation phase at dfget/core/core.go:171, the registration of stream task is necessary for uploader. Here are all the modifications:

• dfget/core/core.go:210: The doDownload method is refactored. One new interface is defined as DownloadTimeoutTask. The interface has two implementations, which are regular downloader and the stream downloader.

• dfget/core/downloader/downloader.go:79: The Start() method of the StreamDownloadTimeoutTask struct will call the RunStream method of the downloader, and then pass the stream reader to the startWriter method.

• dfget/core/downloader/downloader_util.go:55: The startWriter method is used to fetch the pieces from the stream reader concurrently, and then upload the pieces to uploader in order. The logic of the startWriter is referenced from the stream downloading method in CDN manager at supernode/daemon/mgr/cdn/super_writer.go:59.

  The unit test has been added.

3.3 DEGET Uploader

The previous uploader is solely used for the regular downloading method. For the stream uploader, I have added new APIs for it. Apart from that, the cache manager is added to manage the cache pieces.

• dfget/core/api/supernode_api.go:359: A new kind of updating piece status API has been implemented. It is used to change the peice status from SUCCESSFUL to UNCACHED.

• dfget/core/api/uploader_api.go:108: The uploader API RegisterStreamTask is added to register the stream task at cache manager. The initialization of the according entry at cache manager is finshed after the call.

• dfget/core/api/uploader_api.go:127: The uploader API DeliverPieceToUploader is used to handle the piece from downloader to uploader.

• dfget/core/uploader/cache.go:60: The FIFO cache manager is created to control the piece caches. It would store and pop the piece in the style of FIFO. It should be noted that the cache manager is using the receiver window, which has the range of [start, una).

  The unit test has been added.

• TODO: GC for cache manager

3.4 IPC Between DFGET and User

This part demands further discussion. Currently, I propose that the DFGET can directly output the content of the pieces to stdout, and then the user who calls the DFGET command, can redirect the stdout of the DFGET command to the write side of the pipe by generally supported method popen.

After that, the user can directly read from the pipe, and get the successfully downloaded content. Since the method uses the unnamed pipe, the whole process is not related with the file system.

4. Milestone Review

Here is the changes of code lines after the commits: (+2033, -77).

Project Summary

1. Project Deliverables

1.1 Detailed Design

The Overrall Proposal: https://github.com/dragonflyoss/Dragonfly/issues/1436#issue-659970568

Design of Supernode Scheduler: https://github.com/dragonflyoss/Dragonfly/pull/1447#issue-459701444

Design of DFGET downloader, uploader and IPC between user and DFGET: https://github.com/dragonflyoss/Dragonfly/pull/1447#issuecomment-681619384

1.2 Source Code

Github Commits: https://github.com/dragonflyoss/Dragonfly/pull/1447/commits

1.3 Test Document

The tests have been implemented into the test files. And the annotation inside the source files would be necessary for the developer to test.

cmd/dfget/app/root_test.go: https://github.com/dragonflyoss/Dragonfly/pull/1447/files#diff-14e54b172ea500c227d81045b4b132b4R139

dfget/core/downloader/downloader_util_test.go: https://github.com/dragonflyoss/Dragonfly/pull/1447/files#diff-9b3047c6073bec2c5c3a02c87cfce3e2R1

dfget/core/uploader/cache_test.go: https://github.com/dragonflyoss/Dragonfly/pull/1447/files#diff-61f496afc10516997e19f5ac636278f8R1

2. Project Highlights

The highlights of my work is that I have implemented almost all of the downloading in sream mode. The sum of the modification code lines are (2033 + 77) = 2110. And I believe that the stream mode will definitely improve the efficiency of the Dragonfly downloading process in the case of the remote file system.

What’s more, this will be one of the features that Dragonfly owns compared to the traditional P2P systems.

3. Experience

The whole ASoC experience benefits me a lot. First of all, I have done much research work during the application phase, which makes me familiar with the traditional distributed systems. And then, during the development phase, I learnt the system design, concurrent programming in golang, start-of-the-art architecture of Dragonfly and the group work experience with mentors in Alibaba.

Mentors are really good at what they are working for: by the simple discussion, they can always quickly find the weakness of my design and point out the correct direction of the next move.

In a word, it is so lucky for me to be chosen to be part of the 2020 ASoC program.

This blog is used to record the note regarding the detailed implementation of thread-safe LRU strategy in Linux. The requirement behind the note is that I have to develop the same LRU in Dragonfly project for ASoC 2020.

Reference:

• lru_cache.h
• lru_cache.c

This post is going to record the note of design pattern of object-oriented programming. By reading the book, the following ability should be acquired:

• Finding Appropriate Objects: The hard part about OO design is de-composing a system into objects. Design patterns help you identify less-obvious abstractions and the objects that can capture them.
• Determining Object Granularity
• Specifying Object Interfaces:
  The set of all signatures defined by an object’s operations is called the interface to the object.
• Specifying Object Implementations

The reference material is listed below:

• the Github Respository: https://github.com/tmrts/go-patterns
• textbook: Design Patterns: Elements of Reuseable Object-Oriented Software

Christopher Alexander says, “Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice”

Gamma, Erich. Design Patterns (Addison-Wesley Professional Computing Series) (p. 24). Pearson Education. Kindle Edition.

Research done for the design of Dragonfly Streaming Scheduler.

Here P2P Streaming means that the data downloaded by the peer which will then be passed to the target place, is not going to stored in the local file system. The data would be maintained in the memory, and then ditectly send to the target node.

This is necessary for the P2P downloading when the peer server have a cloud disk. In that case, the process of writing the data to the file system of peer server would be limited to the network I/O. But if the P2P streaming is adopted, the writing process would be eliminated.

In this post, I would do research on how to design the scheduler in the supernode/server to distribute the pieces among peer servers.