/[mldonkey]/mldonkey/docs/overnet.txt
ViewVC logotype

Contents of /mldonkey/docs/overnet.txt

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.1 - (show annotations) (download)
Wed Jan 22 23:31:52 2003 UTC (21 years, 3 months ago) by simon_mld
Branch: MAIN
CVS Tags: release-2-04rc1-9, release-2-5-30-12, release-2-5-30-11, release-2-5-30-10, release-2-5-30-17, release-2-5-30-16, release-2-4-3, release-2-4-2, release-2-4-1, release-2-4-0, release-2-4-5, release-2-4-4, release-2-04rc1-7, release-2-5-rc3, release-2-5-rc2, release-2-5-rc1, release-2-04rc3, release-2-04rc2, release-2-5-30-13, release-2-5-30-15, release-2-5-30-14, release-2-4-rc6, release-2-4-rc7, release-2-4-rc4, release-2-4-rc5, release-2-4-rc8, release-2-4-rc9, release-2-04rc1-10, release-3-04rc1, release-2-4-rc12, release-2-5-30-7, release-2-5-30-6, release-2-5-30-5, release-2-5-30-4, release-2-5-30-3, release-2-5-30-2, release-2-5-30-1, release-2-5-4-8, release-2-5-30-9, release-2-5-30-8, unstable-2-02-8, unstable-2-02-9, unstable-2-02-6, unstable-2-02-7, release-2-5-8, release-2-5-9, release-2-5-6, release-2-5-7, release-2-5-4, release-2-5-5, release-2-5-2, release-2-5-3, release-2-5-1, release-2-5-devel-4, release-2-5-devel-6, release-2-04rc1-8, release-2-5-30, release-2-5-29, release-2-5-28, release-2-5-25, release-2-5-24, release-2-5-27, release-2-5-26, release-2-5-21, release-2-5-20, release-2-5-23, release-2-5-22, unstable-2-02-16, release-2-04rc1-6, unstable-2-02-10, unstable-2-02-11, release-2-4-rc10, release-2-4-rc11, release-2-04rc1-15, release-2-04rc1-14, release-2-04rc1-11, release-2-04rc1-12, release-2-5-18, release-2-5-19, release-2-5-10, release-2-5-11, release-2-5-14, release-2-5-15, release-2-5-16, release-2-5-17, release-2-5
Branch point for: release-2-5-28-1, release-2-5-devel, release-2-5-1-patches
File MIME type: text/plain
-

1 The Overnet Protocol
2 --------------------
3
4 The Overnet protocol is divided in two parts: localisation and download.
5 Localisation using UDP messages, while download is done using TCP/IP. The
6 download protocol is mainly the same as in edonkey, except that the
7 Connect/ConnectReply messages miss the server:port fields (ie they are 6
8 bytes shorter).
9
10 In this draft, I focus on the localisation part.
11
12 I Messages Format
13 =================
14
15 1) General Format:
16 ------------------
17
18 UDP messages start with two bytes:
19 byte: 227 eDonkey general magic number for messages
20 byte: the opcode of the message
21 char[]: message args
22
23 2) Opcodes:
24 -----------
25
26 10 OvernetConnect
27 11 OvernetConnectReply
28 12 OvernetPublicize
29 13 OvernetPublicized
30 14 OvernetSearch
31 15 OvernetSearchReply
32 16 OvernetGetSearchResults
33 17 OvernetSearchResult
34 18 OvernetNoResult
35 19 OvernetPublish
36 20 OvernetPublished
37
38 24 OvernetFirewallConnection
39 25 OvernetFirewallConnectionACK
40 26 OvernetFirewallConnectionNACK
41 27 OvernetGetMyIP
42 28 OvernetGetMyIPResult
43 29 OvernetGetMyIPDone
44
45 3) Conventions:
46 ---------------
47
48 int8: one byte
49 int16: two bytes in little endian format
50 int32: four bytes in little endian format
51 ip: an int32, for eg 127.0.0.1 = 127 + 0 << 8 + 0 << 16 + 1 << 24
52 md4: the 16 bytes md4 of the file/client
53
54 peer: an overnet peer in the following format:
55 md4: the peer md4
56 ip: the peer ip
57 int16: the peer port
58 int8: the peer kind/uptime/last seen ?
59
60 tags:
61 int32: <n> number of tags
62 tag[n]: the array of tag (as in edonkey proto)
63
64 4) Messages args:
65 -----------------
66
67 OvernetConnect:
68 ---------------
69 md4: client md4
70 ip: my client ip
71 int16: my client port
72 int8: my client kind/uptime ? 0 is ok
73
74 OvernetConnectReply:
75 --------------------
76 int16: <n> number of peers propagated in this message
77 peer[n]: the peers propagated
78
79 OvernetPublicize:
80 -----------------
81 md4:
82 ip: ip
83 int16: port
84 int8: kind
85
86 OvernetPublicized: no args
87 ------------------
88
89 OvernetSearch:
90 --------------
91 int8: kind of search ?
92 md4: the hash you are looking for
93
94 OvernetSearchReply:
95 -------------------
96 md4: the hash you have asked for
97 int8: <n> number of peers that could be interesting for you
98 peer[n]: the better peers you should ask
99
100 OvernetGetSearchResults:
101 ------------------------
102 md4: the hash you are looking for
103 kind: ?
104 int16: min (the range of results you want)
105 int16: max
106
107 OvernetSearchResult:
108 --------------------
109 md4: the hash you have asked for
110 md4: the hash of the client/file of the result (depends if you look for
111 a file description or for a localisation)
112 tags: the tags describing the result
113
114 OvernetNoResult:
115 ----------------
116 md4: the hash you have asked for
117
118 OvernetPublish:
119 ---------------
120 md4:
121 md4:
122 tags:
123
124 OvernetPublished:
125 -----------------
126 md4: the hash of the published file
127
128 OvernetGetMyIP:
129 ---------------
130 int8: 54
131 int8: 18
132
133 OvernetGetMyIPResult:
134 ---------------------
135 ip: your ip
136
137 OvernetGetMyIPDone: no args
138 -------------------
139
140 OvernetFirewallConnection:
141 --------------------------
142 md4: the md4 of the client you want to access
143 int16: ?
144
145 OvernetFirewallConnectionACK:
146 -----------------------------
147 md4:
148
149 OvernetFirewallConnectionNACK:
150 md4:
151
152 II Client behavior:
153 ===================
154
155 1) Basics:
156 ----------
157
158 In Overnet, the description and localisation of files are stored on peers
159 such that the peer hash is as close as possible to the file hash.
160
161 When a peer wants a file, it tries to ask the peers which have the
162 closest hashs to the file it wants. These peers reply by sending the
163 closest peers they know to the requested hash.
164
165 The distance between two hashs is computed using what Kademlia calls a
166 Xor metric: a xor is computed between the two hashs, and the result
167 hash represents the distance. Two distances are then compared by using
168 a lexicographical order on their bytes (ie first bytes are compared, and
169 if equal, the second bytes, and so on...)
170
171 For this to work, the client needs to maintain a small list of peers it
172 knows, which are as spreaded as possible in the space of possible hashs.
173
174 2) Sequences:
175 -------------
176
177 a) Establishing a "connection" with other peers (booting)
178 ---------------------------------------------------------
179
180 Client Peer
181 ------------------------------------------------
182
183 1: ------- OvernetConnect ------>
184
185 2: <----- OvernetConnectReply ----
186
187 OK, the peer is connected, it can be added to the list of available peers
188 for searches. The peers propagated in the reply can be used for new
189 connections.
190
191 b) Searches
192 -----------
193 From the keywords search, remove all keywords of 1 or 2 chars.
194 Then, hash each keyword separately, and start one search per hash. When you
195 will get results, you will have to filter them so that all keywords are
196 matched, the protocol won't do that for you.
197
198 The search works as follows. You need a set of peers to be searched for the
199 hash. Typically, you take the peers with the hashes with the same first char
200 as the one you search.
201
202 Then, periodically (every 5 seconds ?), you take some peers (2 ? with the
203 closest hashes), and ask them for the hash:
204
205 Client Peer
206 ------------------------------------------------
207
208 1: --------- OvernetSearch ------->
209 1': ------- OvernetGetSearchResults -->
210
211 2: <----- OvernetSearchReply -------
212 2': <----- OvernetSearchResult ------
213 2'': <----- OvernetNoResult -------
214
215 1: query for better peers to ask for the hash
216 2: return new peers which should be queried for the hash
217
218 1': query the peer directly for the hash: do you know it ?
219 2': yes, I know it, here are the files containing this keyword
220 (or the clients sharing this file if you are looking for a file-hash)
221 2'': no, I have no info, try someone else...
222
223 Now, when you have the results, you need to remember what you are
224 searching for:
225 1) keywords: results are file descriptions, closed to edonkey ones
226 (here, maybe some tags are different :)))
227 2) files: you should look in the tags for one called "loc". It contains
228 a string with the format "bcp://ip:port" or "bcp://????:ip:port"
229 containing the source ip and port in ascii.
230
231 Now, you just need to connect and download ...
232
233
234
235

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26