BitTorrent Peer-to-peer controlled by Salt

Markus Kramer

Playlists: 'osc16' videos starting here / audio / related events

Our task is to distribute software to Windows clients. Our network contains slow links and relay-servers, which must be staged up.
Our goals are 1) reduce WAN traffic, 2) allow unlimited size, 3) allow unlimited number of clients in one rollout, and 4) start rollout without delay.

Idea ::
BitTorrent Peer-to-peer (P2P) reduces WAN traffic because if prefers local content over remote content. Beneficially for the concept, all clients are centrally configured: no peer can be a leach. Clients act as a storage resource for other clients, eliminating the need for relay servers.

Realization ::
We found that Salt manages a BitTorrent agent nicely with Salt-states.

Experience/Result ::
We have continuously distributed 2 GB per day to up to 50 Minions for over 2 months. Clients and network are undisturbed, while a Salt-Master on a regular desktop administers 50 Minions effortlessly. We fully meet all of our goals. We observe an increasing and by now high reliability with P2P and Salt (in this order), but glitches still occur in both domains.

Live Demo ::
How to distribute 500 MB, present at 4 clients, to 8 more clients?
During transmission, we will stop the P2P service on some of these 8 clients via salt and then start it again, to simulate clients going temporarily offline.

Next Steps ::
Making Minions 'more active'. Activating and deactivating the P2P service on demand.