IPLD / Multiformats
Last updated
Was this helpful?
Last updated
Was this helpful?
머클트리 포레스트
웹의 진화 - 브라우저는 유지되겠지만 백엔드의 프로토콜은 다양화
데이터로서의 웹 / 데이터 기반의 서비스 / 데이터 종속성 제거 의미
서비스 개발 > 데이터 + 렌더링
This is extracted from the address below
Why do we Need Multiformats?
Ok, I think we need it. But What is it?
This seems great. Tell me how to use it?
Allowing systems to evolve and grow, without introducing breaking changes is important.
Summing up there are a number of problems that we face:
Introducing breaking changes to update systems with better security.
Introducing breaking changes due to some unforeseen issues.
And sometimes we need to make trade-offs when it comes to multiple numbers of options, each having a desired trait, but you can have only one.
The Multiformats Project introduces a set of standards/protocols that embrace this fact and allows multiple protocols to co-exist so that even if there is a breaking change, the ecosystem still supports all the versions of the protocol.
The Multiformats Project is a collection of protocols which aim to future-proof systems, today. They do this mainly by enhancing format values with self-description. This allows interoperability, protocol agility, and helps us avoid lock-in.
The self-describing aspects of the protocols have a few stipulations:
They MUST be in-band (with the value); not out-of-band (in context).
They MUST avoid lock-in and promote extensibility.
They MUST be compact and have a binary-packed representation.
They MUST have a human-readable representation.
Currently, we have the following multiformat protocols:
Multihash: Self-describing hashes
Multiaddr: Self-describing network addresses
Multibase: Self-describing base encodings
Multicodec: Self-describing serialization
Multistream: Self-describing stream network protocols
Multistream-select: Friendly protocol multiplexing.
Multigram(WIP): Self-describing packet network protocols
Multikey: cryptographic keys and artifacts
Each of the projects has its list of implementations in various languages.
You can find a number of multihash implementations in multiple languages.
/ip4/127.0.0.1/udp/9090/quic
/ip6/::1/tcp/3217
/ip4/127.0.0.1/tcp/80/http/baz.jpg
/dns4/foo.com/tcp/80/http/bar/baz.jpg
/dns6/foo.com/tcp/443/httpsmulticodec is a self-describing multiformat, it wraps other formats with a tiny bit of self-description. A multicodec identifier is a varint.
A chunk of data identified by multicodec will look like this:
<multicodec><encoded-data>
# To reduce the cognitive load, we sometimes might write the same line as:
<mc><data>Multicodec uses “protocol tables” to agree upon the mapping from one multicodec code. These tables can be application specific, though — like with other multiformats — we will keep a globally agreed upon table with common protocols and formats.
In order to enable self-descriptive data formats or streams that can be dynamically described, without the formal set of adding a binary packed code to a table, we have multistream, so that applications can adopt multiple data formats for their streams and with that create different protocols.
Multicodecs are self-describing protocol/encoding streams. (Note that a file is a stream). It’s designed to address the perennial problem:
I have a bitstring, what codec is the data coded with?
To decode an incoming stream of data, a program must either
know the format of the data a priori, or
learn the format from the data itself.
(1) precludes running protocols that may provide one of many kinds of formats without prior agreement on which. multistream makes (2) neat using self-description.
Moreover, this self-description allows straightforward layering of protocols without having to implement support in the parent (or encapsulating) one.
multistream is a self-describing multiformat, it wraps other formats with a tiny bit of self-description:
<varint-len>/<codec>\n<encoded-data>For example, let’s encode a JSON doc:
// encode some json
const buf = new Buffer(JSON.stringify({ hello: 'world' }))const prefixedBuf = multistream.addPrefix('json', buf) // prepends multicodec ('json')
console.log(prefixedBuf)
// <Buffer 06 2f 6a 73 6f 6e 2f 7b 22 68 65 6c 6c 6f 22 3a 22 77 6f 72 6c 64 22 7d>console.log(prefixedBuf.toString('hex'))
// 062f6a736f6e2f7b2268656c6c6f223a22776f726c64227d// let's get the Codec and then get the data backconst codec = multicodec.getCodec(prefixedBuf)
console.log(codec)
// jsonconsole.log(multistream.rmPrefix(prefixedBuf).toString())
// "{ \"hello\": \"world\" }So, buf is:
hex: 062f6a736f6e2f7b2268656c6c6f223a22776f726c64227d
ascii: /json\n"{\"hello\":\"world\"}"Note that on the ASCII version, the varint at the beginning is not being represented, you should account that.
You can find multistream-select tutorial here.
