作者:谢小路 链接:https://www.jianshu.com/p/8cc7ed063e6e 來源:简书
大家好,我叫谢伟,是一名程序员。
下面结合我的经历和见闻,讲述下一名非科班程序员的成长过程:
一般初学者确定一个方向,比如web 后端、前端等,会选择一门编程语言深入下去,比如后端java、python、go等。通过项目不断练习编程语言和编程思维,知道如何将需求实现出来。一段时间后,有可能算是某一阶段的瓶颈,希望写出更好的代码,除了继续做项目之外,更好的方式是阅读某一个库或者某一项目的源代码,从源代码里学习一些编程的处理方式,之后借鉴到自己的项目中。突破瓶颈,继续精进技能。
一般的软件构建过程是这样的:
本节的主题是:如何阅读源代码?
开源领域,值得学习的东西太多了,你应该明确知道你需要解决的问题是什么,才能针对性的对某一项目或者某一库进行源代码的阅读。
go-restful
是用于构建REST-style web
服务的golang
包。
在这之前我们需要了解下 HTTP
协议、Web 客户端、服务端。
这些知识和我们访问网址获取到的信息息息相关。
我们在浏览器中输入:URL
的整体过程如下:
我们关注里面的:HTTP Request
和 HTTP Response
随意找个网页查看源代码看看:
HTTP Request.png
HTTP 协议:HTTP Request
1GET /u/58f0817209aa HTTP/1.1
2Host: www.jianshu.com
3Connection: keep-alive
4Pragma: no-cache
5Cache-Control: no-cache
6Upgrade-Insecure-Requests: 1
7User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36
8Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
9Referer: https://www.jianshu.com/
10Accept-Encoding: gzip, deflate, br
11Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
主要包括:
HTTP 协议 HTTP Response
1HTTP/1.1 200 OK
2Date: Sun, 20 May 2018 03:19:36 GMT
3Server: Tengine
4Content-Type: text/html; charset=utf-8
5Transfer-Encoding: chunked
6X-Frame-Options: DENY
7X-XSS-Protection: 1; mode=block
8X-Content-Type-Options: nosniff
9Content-Security-Policy: script-src 'self' 'unsafe-inline' 'unsafe-eval' *.jianshu.com *.jianshu.io api.geetest.com static.geetest.com dn-staticdown.qbox.me zz.bdstatic.com *.google-analytics.com hm.baidu.com push.zhanzhang.baidu.com res.wx.qq.com qzonestyle.gtimg.cn as.alipayobjects.com ;style-src 'self' 'unsafe-inline' *.jianshu.com *.jianshu.io api.geetest.com static.geetest.com ;
10ETag: W/"4d22fb2fcef7cdb3f874a6b4960ff2ae"
11Cache-Control: max-age=0, private, must-revalidate
12Set-Cookie: locale=zh-CN; path=/
13Set-Cookie: _m7e_session=708ecf714930ebc19da67ae3141bd6c0; path=/; expires=Sun, 20 May 2018 09:19:36 -0000; secure; HttpOnly
14X-Request-Id: c61a268c-896f-4e03-afbe-2547db04943d
15X-Runtime: 0.137573
16Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
17Content-Encoding: gzip
18X-Via: 1.1 PSfjfzdx2wn96:6 (Cdn Cache Server V2.0), 1.1 jsyz89:1 (Cdn Cache Server V2.0)
19Connection: keep-alive
20X-Dscp-Value: 0
主要包括:
所以关于设计 restful api 的主体部分包括这些:
鉴于上面的知识点,我们如果使用内置的golang 包,处理 http 信息会这么做:
1func Downloader(url string) ([]byte, error) {
2 var (
3 req *http.Request
4 err error
5 )
6 if req, err = http.NewRequest("GET", url, nil); err != nil {
7 return nil, ErrorHttpRequest
8 }
9
10 client := http.DefaultClient
11 req.Header.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36")
12 var (
13 resp *http.Response
14 )
15 if resp, err = client.Do(req); err != nil {
16 return nil, ErrorHttpResponse
17 }
18
19 defer resp.Body.Close()
20
21 return ioutil.ReadAll(resp.Body)
22}
查看下源代码 net/http 库中的 http.Request 和 http.Response 都有些什么?
1type Request struct {
2 // Method specifies the HTTP method (GET, POST, PUT, etc.).
3 // For client requests an empty string means GET.
4 Method string
5
6 // URL specifies either the URI being requested (for server
7 // requests) or the URL to access (for client requests).
8 //
9 // For server requests the URL is parsed from the URI
10 // supplied on the Request-Line as stored in RequestURI. For
11 // most requests, fields other than Path and RawQuery will be
12 // empty. (See RFC 2616, Section 5.1.2)
13 //
14 // For client requests, the URL's Host specifies the server to
15 // connect to, while the Request's Host field optionally
16 // specifies the Host header value to send in the HTTP
17 // request.
18 URL *url.URL
19
20 // The protocol version for incoming server requests.
21 //
22 // For client requests these fields are ignored. The HTTP
23 // client code always uses either HTTP/1.1 or HTTP/2.
24 // See the docs on Transport for details.
25 Proto string // "HTTP/1.0"
26 ProtoMajor int // 1
27 ProtoMinor int // 0
28
29 // Header contains the request header fields either received
30 // by the server or to be sent by the client.
31 //
32 // If a server received a request with header lines,
33 //
34 // Host: example.com
35 // accept-encoding: gzip, deflate
36 // Accept-Language: en-us
37 // fOO: Bar
38 // foo: two
39 //
40 // then
41 //
42 // Header = map[string][]string{
43 // "Accept-Encoding": {"gzip, deflate"},
44 // "Accept-Language": {"en-us"},
45 // "Foo": {"Bar", "two"},
46 // }
47 //
48 // For incoming requests, the Host header is promoted to the
49 // Request.Host field and removed from the Header map.
50 //
51 // HTTP defines that header names are case-insensitive. The
52 // request parser implements this by using CanonicalHeaderKey,
53 // making the first character and any characters following a
54 // hyphen uppercase and the rest lowercase.
55 //
56 // For client requests, certain headers such as Content-Length
57 // and Connection are automatically written when needed and
58 // values in Header may be ignored. See the documentation
59 // for the Request.Write method.
60 Header Header
61
62 // Body is the request's body.
63 //
64 // For client requests a nil body means the request has no
65 // body, such as a GET request. The HTTP Client's Transport
66 // is responsible for calling the Close method.
67 //
68 // For server requests the Request Body is always non-nil
69 // but will return EOF immediately when no body is present.
70 // The Server will close the request body. The ServeHTTP
71 // Handler does not need to.
72 Body io.ReadCloser
73
74 // GetBody defines an optional func to return a new copy of
75 // Body. It is used for client requests when a redirect requires
76 // reading the body more than once. Use of GetBody still
77 // requires setting Body.
78 //
79 // For server requests it is unused.
80 GetBody func() (io.ReadCloser, error)
81
82 // ContentLength records the length of the associated content.
83 // The value -1 indicates that the length is unknown.
84 // Values >= 0 indicate that the given number of bytes may
85 // be read from Body.
86 // For client requests, a value of 0 with a non-nil Body is
87 // also treated as unknown.
88 ContentLength int64
89
90 // TransferEncoding lists the transfer encodings from outermost to
91 // innermost. An empty list denotes the "identity" encoding.
92 // TransferEncoding can usually be ignored; chunked encoding is
93 // automatically added and removed as necessary when sending and
94 // receiving requests.
95 TransferEncoding []string
96
97 // Close indicates whether to close the connection after
98 // replying to this request (for servers) or after sending this
99 // request and reading its response (for clients).
100 //
101 // For server requests, the HTTP server handles this automatically
102 // and this field is not needed by Handlers.
103 //
104 // For client requests, setting this field prevents re-use of
105 // TCP connections between requests to the same hosts, as if
106 // Transport.DisableKeepAlives were set.
107 Close bool
108
109 // For server requests Host specifies the host on which the
110 // URL is sought. Per RFC 2616, this is either the value of
111 // the "Host" header or the host name given in the URL itself.
112 // It may be of the form "host:port". For international domain
113 // names, Host may be in Punycode or Unicode form. Use
114 // golang.org/x/net/idna to convert it to either format if
115 // needed.
116 //
117 // For client requests Host optionally overrides the Host
118 // header to send. If empty, the Request.Write method uses
119 // the value of URL.Host. Host may contain an international
120 // domain name.
121 Host string
122
123 // Form contains the parsed form data, including both the URL
124 // field's query parameters and the POST or PUT form data.
125 // This field is only available after ParseForm is called.
126 // The HTTP client ignores Form and uses Body instead.
127 Form url.Values
128
129 // PostForm contains the parsed form data from POST, PATCH,
130 // or PUT body parameters.
131 //
132 // This field is only available after ParseForm is called.
133 // The HTTP client ignores PostForm and uses Body instead.
134 PostForm url.Values
135
136 // MultipartForm is the parsed multipart form, including file uploads.
137 // This field is only available after ParseMultipartForm is called.
138 // The HTTP client ignores MultipartForm and uses Body instead.
139 MultipartForm *multipart.Form
140
141 // Trailer specifies additional headers that are sent after the request
142 // body.
143 //
144 // For server requests the Trailer map initially contains only the
145 // trailer keys, with nil values. (The client declares which trailers it
146 // will later send.) While the handler is reading from Body, it must
147 // not reference Trailer. After reading from Body returns EOF, Trailer
148 // can be read again and will contain non-nil values, if they were sent
149 // by the client.
150 //
151 // For client requests Trailer must be initialized to a map containing
152 // the trailer keys to later send. The values may be nil or their final
153 // values. The ContentLength must be 0 or -1, to send a chunked request.
154 // After the HTTP request is sent the map values can be updated while
155 // the request body is read. Once the body returns EOF, the caller must
156 // not mutate Trailer.
157 //
158 // Few HTTP clients, servers, or proxies support HTTP trailers.
159 Trailer Header
160
161 // RemoteAddr allows HTTP servers and other software to record
162 // the network address that sent the request, usually for
163 // logging. This field is not filled in by ReadRequest and
164 // has no defined format. The HTTP server in this package
165 // sets RemoteAddr to an "IP:port" address before invoking a
166 // handler.
167 // This field is ignored by the HTTP client.
168 RemoteAddr string
169
170 // RequestURI is the unmodified Request-URI of the
171 // Request-Line (RFC 2616, Section 5.1) as sent by the client
172 // to a server. Usually the URL field should be used instead.
173 // It is an error to set this field in an HTTP client request.
174 RequestURI string
175
176 // TLS allows HTTP servers and other software to record
177 // information about the TLS connection on which the request
178 // was received. This field is not filled in by ReadRequest.
179 // The HTTP server in this package sets the field for
180 // TLS-enabled connections before invoking a handler;
181 // otherwise it leaves the field nil.
182 // This field is ignored by the HTTP client.
183 TLS *tls.ConnectionState
184
185 // Cancel is an optional channel whose closure indicates that the client
186 // request should be regarded as canceled. Not all implementations of
187 // RoundTripper may support Cancel.
188 //
189 // For server requests, this field is not applicable.
190 //
191 // Deprecated: Use the Context and WithContext methods
192 // instead. If a Request's Cancel field and context are both
193 // set, it is undefined whether Cancel is respected.
194 Cancel <-chan struct{}
195
196 // Response is the redirect response which caused this request
197 // to be created. This field is only populated during client
198 // redirects.
199 Response *Response
200
201 // ctx is either the client or server context. It should only
202 // be modified via copying the whole Request using WithContext.
203 // It is unexported to prevent people from using Context wrong
204 // and mutating the contexts held by callers of the same request.
205 ctx context.Context
206}
1type Response struct {
2 Status string // e.g. "200 OK"
3 StatusCode int // e.g. 200
4 Proto string // e.g. "HTTP/1.0"
5 ProtoMajor int // e.g. 1
6 ProtoMinor int // e.g. 0
7
8 // Header maps header keys to values. If the response had multiple
9 // headers with the same key, they may be concatenated, with comma
10 // delimiters. (Section 4.2 of RFC 2616 requires that multiple headers
11 // be semantically equivalent to a comma-delimited sequence.) Values
12 // duplicated by other fields in this struct (e.g., ContentLength) are
13 // omitted from Header.
14 //
15 // Keys in the map are canonicalized (see CanonicalHeaderKey).
16 Header Header
17
18 // Body represents the response body.
19 //
20 // The http Client and Transport guarantee that Body is always
21 // non-nil, even on responses without a body or responses with
22 // a zero-length body. It is the caller's responsibility to
23 // close Body. The default HTTP client's Transport does not
24 // attempt to reuse HTTP/1.0 or HTTP/1.1 TCP connections
25 // ("keep-alive") unless the Body is read to completion and is
26 // closed.
27 //
28 // The Body is automatically dechunked if the server replied
29 // with a "chunked" Transfer-Encoding.
30 Body io.ReadCloser
31
32 // ContentLength records the length of the associated content. The
33 // value -1 indicates that the length is unknown. Unless Request.Method
34 // is "HEAD", values >= 0 indicate that the given number of bytes may
35 // be read from Body.
36 ContentLength int64
37
38 // Contains transfer encodings from outer-most to inner-most. Value is
39 // nil, means that "identity" encoding is used.
40 TransferEncoding []string
41
42 // Close records whether the header directed that the connection be
43 // closed after reading Body. The value is advice for clients: neither
44 // ReadResponse nor Response.Write ever closes a connection.
45 Close bool
46
47 // Uncompressed reports whether the response was sent compressed but
48 // was decompressed by the http package. When true, reading from
49 // Body yields the uncompressed content instead of the compressed
50 // content actually set from the server, ContentLength is set to -1,
51 // and the "Content-Length" and "Content-Encoding" fields are deleted
52 // from the responseHeader. To get the original response from
53 // the server, set Transport.DisableCompression to true.
54 Uncompressed bool
55
56 // Trailer maps trailer keys to values in the same
57 // format as Header.
58 //
59 // The Trailer initially contains only nil values, one for
60 // each key specified in the server's "Trailer" header
61 // value. Those values are not added to Header.
62 //
63 // Trailer must not be accessed concurrently with Read calls
64 // on the Body.
65 //
66 // After Body.Read has returned io.EOF, Trailer will contain
67 // any trailer values sent by the server.
68 Trailer Header
69
70 // Request is the request that was sent to obtain this Response.
71 // Request's Body is nil (having already been consumed).
72 // This is only populated for Client requests.
73 Request *Request
74
75 // TLS contains information about the TLS connection on which the
76 // response was received. It is nil for unencrypted responses.
77 // The pointer is shared between responses and should not be
78 // modified.
79 TLS *tls.ConnectionState
80}
可以看出这两个结构体内存在着我们之前分析的那些点。
如果只使用内置的 net/http 的包如何启动一个web 服务?
1package main
2
3import (
4 "fmt"
5 "net/http"
6)
7
8func Say(resp http.ResponseWriter, req *http.Request) {
9 req.ParseForm()
10 fmt.Println(req.URL.Host, "-", req.URL.Path, "-", req.Form)
11 fmt.Fprintf(resp, "hello world")
12}
13
14func main() {
15 http.HandleFunc("/user/hello", Say)
16 http.ListenAndServe(":8080", nil)
17}
访问:localhost:8080/user/hello
返回响应值:"hello world"
上文中:URL、和响应值response,我们在代码中进行了处理。同样的我们访问真实的网址, 比如 https://www.baidu.com
则是百度的服务器端代码进行了处理。
上文中大概知道了构建 restful api 相关的一些 http 协议的知识, 和内置的库 net/http 的基本使用方法。
但别忘了我们的主题是:阅读 go-restful 的源代码。
首先,我们应该根据官方文档学会基本的使用:
1package main
2
3import (
4 "fmt"
5 "log"
6 "net/http"
7
8 "github.com/emicklei/go-restful"
9)
10
11
12
13type User struct {
14 Name string
15 Age string
16 ID []int
17}
18
19type UserResource struct {
20 // normally one would use DAO (data access object)
21 users map[string]User
22}
23
24// WebService creates a new service that can handle REST requests for User resources.
25func (u UserResource) WebService() *restful.WebService {
26 ws := new(restful.WebService)
27 ws.
28 Path("/users").
29 Consumes(restful.MIME_XML, restful.MIME_JSON).
30 Produces(restful.MIME_JSON, restful.MIME_XML) // you can specify this per route as well
31
32 ws.Route(ws.GET("/").To(u.findAllUsers).
33 // docs
34 Doc("get all users").
35 Writes([]User{}).
36 Returns(200, "OK", []User{}))
37
38 ws.Route(ws.GET("/{user-id}").To(u.findUser).
39 // docs
40 Doc("get a user").
41 Param(ws.PathParameter("user-id", "identifier of the user").DataType("integer").DefaultValue("1")).
42 Writes(User{}). // on the response
43 Returns(200, "OK", User{}).
44 Returns(404, "Not Found", nil))
45
46 return ws
47}
48
49// GET http://localhost:8080/users
50//
51func (u UserResource) findAllUsers(request *restful.Request, response *restful.Response) {
52 list := []User{}
53 for _, each := range u.users {
54 list = append(list, each)
55 }
56 response.WriteEntity(list)
57}
58
59func (u UserResource) findUser(request *restful.Request, response *restful.Response) {
60 id := request.PathParameter("user-id")
61 usr := u.users[id]
62 if len(usr.ID) == 0 {
63 response.WriteErrorString(http.StatusNotFound, "User could not be found.")
64 } else {
65 response.WriteEntity(usr)
66 }
67}
68
69func main() {
70 type APIServer struct {
71 Container *restful.Container
72 }
73 u := UserResource{map[string]User{}}
74 u.users["xiewei"] = User{
75 Name: "xiewei",
76 Age: "20",
77 ID: []int{1, 2, 3, 4},
78 }
79 apiServer := &APIServer{
80 Container: restful.DefaultContainer.Add(u.WebService()),
81 }
82
83 log.Printf("start listening on localhost:9990")
84 log.Fatal(http.ListenAndServe(":9990", apiServer.Container))
85}
访问:localhost:9990/users
1HTTP/1.1 200 OK
2Content-Type: application/json
3Date: Sun, 20 May 2018 04:21:29 GMT
4Content-Length: 92
5
6[
7 {
8 "Name": "xiewei",
9 "Age": "20",
10 "ID": [
11 1,
12 2,
13 3,
14 4
15 ]
16 }
17 ]
访问:localhost:9990/users/xiewei
1HTTP/1.1 200 OK
2Content-Type: application/json
3Date: Sun, 20 May 2018 04:21:29 GMT
4Content-Length: 92
5
6[
7 {
8 "Name": "xiewei",
9 "Age": "20",
10 "ID": [
11 1,
12 2,
13 3,
14 4
15 ]
16 }
17 ]
访问:localhost:9990/users/xiewei2
1HTTP/1.1 404 Not Found
2Date: Sun, 20 May 2018 04:22:59 GMT
3Content-Length: 24
4Content-Type: text/plain; charset=utf-8
5
6User could not be found.
通过这个简单的例子,我们大概能够使用 go-restful 了。
无外乎还是操作:http.Request、http.Response, 上述例子的核心是:findAllUsers
和 findUser
这个两个函数,具体的返回值、状态码什么的都是由这两个函数定义。其他的都是一些路由的定义、定义生产者和消费者格式、启动指定端口的web 服务。
restful-flow.png
1func ListenAndServe(addr string, handler Handler) error {
2 server := &Server{Addr: addr, Handler: handler}
3 return server.ListenAndServe()
4}
能看出函数的入口是:Handler 接口
1type Handler interface {
2 ServeHTTP(ResponseWriter, *Request)
3}
httpServer 包含 container .
1log.Fatal(http.ListenAndServe(":9990", apiServer.Container))
一个 Container 包含多个 WebService
1type Container struct {
2 webServicesLock sync.RWMutex
3 webServices []*WebService
4 ServeMux *http.ServeMux
5 isRegisteredOnRoot bool
6 containerFilters []FilterFunction
7 doNotRecover bool // default is true
8 recoverHandleFunc RecoverHandleFunction
9 serviceErrorHandleFunc ServiceErrorHandleFunction
10 router RouteSelector // default is a CurlyRouter (RouterJSR311 is a slower alternative)
11 contentEncodingEnabled bool // default is false
12}
container 实现的了Handler 接口
1func (c *Container) ServeHTTP(httpwriter http.ResponseWriter, httpRequest *http.Request) {
2 c.ServeMux.ServeHTTP(httpwriter, httpRequest)
3}
一个 webservice 包含多个Route
1type WebService struct {
2 rootPath string
3 pathExpr *pathExpression // cached compilation of rootPath as RegExp
4 routes []Route
5 produces []string
6 consumes []string
7 pathParameters []*Parameter
8 filters []FilterFunction
9 documentation string
10 apiVersion string
11
12 typeNameHandleFunc TypeNameHandleFunction
13
14 dynamicRoutes bool
15
16 // protects 'routes' if dynamic routes are enabled
17 routesLock sync.RWMutex
18}
一个 Route 包含HTTP 协议协议相关的HTTP Request 、HTTP Reponse 、方法等处理
1type Route struct {
2 Method string
3 Produces []string
4 Consumes []string
5 Path string // webservice root path + described path
6 Function RouteFunction
7 Filters []FilterFunction
8 If []RouteSelectionConditionFunction
9
10 // cached values for dispatching
11 relativePath string
12 pathParts []string
13 pathExpr *pathExpression // cached compilation of relativePath as RegExp
14
15 // documentation
16 Doc string
17 Notes string
18 Operation string
19 ParameterDocs []*Parameter
20 ResponseErrors map[int]ResponseError
21 ReadSample, WriteSample interface{} // structs that model an example request or response payload
22
23 // Extra information used to store custom information about the route.
24 Metadata map[string]interface{}
25
26 // marks a route as deprecated
27 Deprecated bool
28}
具体的处理函数是:RouteFunction
1type RouteFunction func(*Request, *Response)
再回过来看一下,我们的代码是怎么处理的:
1log.Fatal(http.ListenAndServe(":9990", apiServer.Container))
1 apiServer := &APIServer{
2 Container: restful.DefaultContainer.Add(u.WebService()),
3 }
1func (u UserResource) WebService() *restful.WebService {
2 ws := new(restful.WebService)
3 ws.
4 Path("/users").
5 Consumes(restful.MIME_XML, restful.MIME_JSON).
6 Produces(restful.MIME_JSON, restful.MIME_XML) // you can specify this per route as well
7
8 ws.Route(ws.GET("/").To(u.findAllUsers).
9 // docs
10 Doc("get all users").
11 Writes([]User{}).
12 Returns(200, "OK", []User{}))
13
14 ws.Route(ws.GET("/{user-id}").To(u.findUser).
15 // docs
16 Doc("get a user").
17 Param(ws.PathParameter("user-id", "identifier of the user").DataType("integer").DefaultValue("1")).
18 Writes(User{}). // on the response
19 Returns(200, "OK", User{}).
20 Returns(404, "Not Found", nil))
21
22 return ws
23}
好,上面的大致处理流程我们已经梳理清楚。
内置库内存在很多的接口,对接口的实现,不断的对内置库的扩展,有可能就重新发明了一个热门的轮子。
go-restful 库便是对内置库 net/http 的扩展。
总结: 阅读源代码首先你需要明确解决的问题是什么,其次你会使用该项目的Demo 或者多个示例,然后你需要根据源代码梳理源代码流程,最后由抄的过程转变为借鉴使用的过程。
版权申明:内容来源网络,版权归原创者所有。除非无法确认,我们都会标明作者及出处,如有侵权烦请告知,我们会立即删除并表示歉意。谢谢。