What is the relationship between DEX coin and DEXP2P network?

The AtomicDEX application uses its backend API served by daemon named marketmaker. The earlier version was written in C language by jl777, the marketmaker version 1. This new marketmaker version 2, in short mm2, which is the backend API service for the graphical application such as AtomicDEX is built from scratch in modern and popular programming, Rust language.

I had some doubts and questions of my own too, so I asked jl777 in #ask-jl777 channel in Komodo Discord. Sharing that chat log here. It has better answers.

grewalsatinder — 02/14/2020 at 8:14 PM
@jl777c mm2 uses libtorrent to create its independent network to communicate between nodes, where the orderbook and ordermatching is done.
with new network ID, it is also scalable.

So, in that sense why we needed another scalable solution such as dexp2p?
what difference is the main difference in relation to mm2 scaling in comparison to dexp2p ?
Trying to understand the reason why dexp2p was started. :slight_smile:

jl777c — 02/14/2020 at 8:15 PM
libtorrent is slow

grewalsatinder — 02/14/2020 at 8:16 PM
isn’t the komodod also using libtorrent lib in it’s core for p2p layer? :thinking:
if not, what is it using? and how it’s faster?

jl777c — 02/14/2020 at 8:16 PM
mm2 used a very inefficient method to broadcast as it was just a placeholder, so it needed a new p2p network layer. after seeing discussions about it for 6 months, i decided to just write it as i knew i could in a few weeks
bitcoin doesnt use libtorrent that i know of
i use the same peer networking as bitcoin code, it is the actual same code that is handling a new p2p message
its faster as i made it faster
https://medium.com/@jameslee777/fully-decentralized-p2p-orderbook-and-more-a9843294da01

I had thought that there would already be an easy to use module to spawn a p2p network that would synchronize any sort of data that was…

grewalsatinder — 02/14/2020 at 8:18 PM
okay, from bitcoin white-paper I thought it’s still using libtorrent for p2p.
is that some custom p2p or anything I can read on what bitcoin’s p2p layer is built upon?

jl777c — 02/14/2020 at 8:19 PM
i think it is libevent, just tcp connections
and its p2p messages

grewalsatinder — 02/14/2020 at 8:19 PM
oh! that’s why it’s faster!

jl777c — 02/14/2020 at 8:20 PM
https://en.bitcoin.it/wiki/Protocol_documentation
direct peer to peer dedicated tcp/ip connections
send a message directly to a peer
given that, and only that, to get an efficient broadcast without a lot of redundant traffic and having it decent performance…
well it did take me a few weeks to get working, maybe someone else would have taken a lot longer to achieve the current level of performance and efficiency

basically the mm2 orderbook matching bogged down with 100+ users, but it wasnt coded to handle more than test amounts of traffic. someone had to make a more efficient orderbook/ordermatching, so i did. not sure what the mystery is, unless you missed the atomicDEX stress test

that is the origin of dexp2p, but i made it generic so it can be used for any usecase that needs to have efficient message broadcasting over p2p network, so it supports file sync, even video streaming. with the tagA/tagB and e2e encrypted messages it can easily be used by a dapp to implement a notarizer or a discord equivalent, or subatomic, or …

blockchains are non-realtime and have an unpredictable blocktime, 10 minute wait for notarization, basically not usable for realtime cases. dexp2p provides the realtime response and it is inside the coin daemon, so it can do subatomics (dexp2p directly linked to payments)

grewalsatinder — 02/14/2020 at 8:27 PM
I missed the tests I think. I was busy somewhere else, and was only glancing here on channels for some months occasionally.

jl777c — 02/14/2020 at 8:28 PM
it basically melted down during the first minutes
so clearly a solution was needed. maybe i overdid it and made a new highly flexible decentralized messaging solution, but it works quite well for orderbook, ordermatching and trade executions

grewalsatinder — 02/14/2020 at 8:33 PM
Thanks :)
clears things much better 👍
can you also share any specific challenges faced stabilising dexp2p to handle big data packets on p2p network?

jl777c — 02/14/2020 at 8:35 PM
i implemented a txpow on a per packet basis, the larger the packet the more txpow needed for more data, so it gets very expensive to send big packets

the file transfers use 10000 byte packets and it is reassembled into slices and slices into file

once the low level messaging worked reliably, the file level was just a couple days work, most of it fiddling with getting streaming working. no real hurdles

though there is a bug still with streaming, so maybe that is the hurdle

grewalsatinder — 02/14/2020 at 8:38 PM
wow! so the txpow, which was used in DiceCC, and I think also used in AdoptedPoW also helped with dexp2p… :thinking:

jl777c — 02/14/2020 at 8:39 PM
it is a different variant of txpow
but yes, the concept of using pow per message is quite useful, but bitmessage did this years ago

grewalsatinder — 02/14/2020 at 8:39 PM
ah yes…. with bitmessage it clears more :slight_smile:

jl777c — 02/14/2020 at 8:40 PM
there is also an anonsend rpc call that sends to everybody

grewalsatinder — 02/14/2020 at 8:40 PM
need bit more to know on txpow
is that also like bitcoin’s PoW? using SHA256 to do PoW per transaction or event message sending data packet over network?

jl777c — 02/14/2020 at 8:41 PM
but only the destination can decrypt it
every data packet gets a hash sha256(curve25519(sha256(message)))

the curve25519 is to make it fpga resistant and also there are no ASIC for the above hash that i know of and unlikely for there to be anytime soon

grewalsatinder — 02/14/2020 at 8:44 PM
and I guess it won’t be a big deal to change it, since it’s jus p2p network based and nothing related to blockchain and avoiding hardfork concerns, in case some naughty fpga programmer figures it out and make that algo work on fpga. Please correct me if any assumption in this line is wrong.

jl777c — 02/14/2020 at 8:47 PM
there are “network hardforks” that would be needed if the basic algos are changed, but that is a matter of updating to new version, there is no blockchain to sync. also, i designed it so that dapps can track if their messages are getting through, and if not, just raise the priority level. that wont help against an fpga, but against a spam attack, it can make the spam exponentially more expensive to make

it also allows someone who really, really wants to broadcast their packet to generate a higher priority one. when the network is saturated, it goes into a different mode and routes the highest priority packets first, the lowest ones it might just totally ignore

jl777c — 02/14/2020 at 8:54 PM
every coin can get its own -dexp2p network within its set of peers, just add the -dexp2p=2