This milestone serves the purpose of setting the methodology plan for the rest of the project. The milestone includes reviewing of the suggested methodology and the code that needs to be implemented in order to get the right results. This is the right time to make amendments to the suggested methodology. Given that there is a suggested methodology, this Milestone does not come with a reward, but should be part of the work.

Methodology details

<h3>Initial results/PoC:</h3>These results and the initial development help in order to judge whether we’re on the right track, or if adjustments need to be made. The purpose of this milestone is to identify whether we have enough detail, or if more detail needs to be integrated in the system, whether we will be able to extract the information we need by the final software and results, or if any readjustment needs to be done to the strategy followed. The PoC should show all the steps in the methodology, but not necessarily deployed at scale. Ideally, there should be initial results from a limited set of publish operations.

Initial Results / Proof of Concept

<h3>Reproducible Experiments:</h3>Anyone should be able to reproduce the experiments done by the team. This is especially important for this project, as the dynamics and structure of the IPFS network changes over time and hence, engineering teams, or community members should be able to re-run the experiments in the future. <ol><li>Any code developed for this project should be migrated, or at least linked from the main GH repository: https://github.com/protocol/network-measurements.</li><li>The code should include detailed documentation/instructions on how to use the developed software, as well as contact details of the team.</li></ol> This milestone is expected before the end of the project and is not expected to be the final software. Updates can be done to the software and documentation after this milestone too. The milestone is needed in order for our team to carefully review the code, the code comments, as well as the HOWTO guide and provide feedback/ask for revisions in case things are not clear.

Reproducible Experiments & Stable Results

<h3>Final Report:</h3>We expect a final report that details the main findings which address the Request for Measurements (RFM), the methodology used,&nbsp;as well as any conclusions made and open issues for future investigation. The conclusions should be accompanied by plots that justify the findings. The report is not expected to be long - think of a relatively detailed blogpost. The Final Report should be posted on the GH repository in the designated folder.

Final Report

Provider records are replicated in the system to&nbsp;<code style="background-color: var(--color-neutral-muted);">k=20</code>&nbsp;peers and are re-provided after 12hrs in the hope that, despite network churn, at least one of them will be alive to provide the record throughout the 12hr interval. However, we have not tested whether provider records indeed stay alive for 12hrs. In addition, we have found that the network has very high churn rate (at times in the order of 50% per hour). Back of the envelope calculation suggests that with this rate of churn all 20 peers are likely to have gone offline after only 6hrs. In turn, this suggests that records might be unreachable for ~6hrs, which should be considered&nbsp;unacceptable. Of course, peers do not only leave, but also come back - our results suggest that the interarrival time is of similar levels as the churn rate - so it is likely that records become available again. Despite the above argumentation, the network seems to work fine (although there have been reports of content being unreachable), so there are three things that might be happening:<ul><li>Hydras help tremendously.&nbsp;Hydras should not be seen as a component of the core architecture and more like a supporting component. So we should not assume full dependence on hydras and have the settings configured correctly.</li><li>Peers churn, but come back online very often.&nbsp;Peers go offline very often, but come back online again too, so if interarrival time is small enough, this helps with keeping records alive.</li><li>Churn rate calculation is not accurate.&nbsp;The actual churn rate is not as calculated and we're missing something.</li></ul> Additional context can be found <a href="https://pl-strflt.notion.site/Provider-Record-Liveness-0b777eb2a3164c67974097eb97be23ad" rel="noopener noreferrer" target="_blank">here</a>.

Provider records are needed in IPFS in order to keep published content live and discoverable. We want to investigate whether provider records stay live for long enough before they are republished, given the high churn of network peers.

IPFS Provider Record Liveness

Active workers:

Jump to section:

IPFS Provider Record Liveness

Problem Description

Solution Requirements

Milestones

Other Information