#jam:polkadot.io

2024-04-20 02:08 johnjiao123:

2024-04-20 02:09 johnjiao123:

2024-05-30 21:48 nradko: Hey everyone, are the lectures explaining gray paper recorded and available online?

2024-05-31 10:54 gav: > <@nradko:matrix.org> Hey everyone, > are the lectures explaining gray paper recorded and available online? They are recorded but not online yet. Will be soon.

2024-05-31 10:55 nradko: > <@gav:polkadot.io> They are recorded but not online yet. Will be soon. can't wait.

2024-06-02 19:50 purpletentacle: A couple of quick questions: - Are there any conformance tests already public? (as per rule 15) - is FFI for external libs acceptable (e.g. crypto from a rust lib) or while libs are acceptable, they need to be native to the chosen language? this was written in the w3f page but does not seem to be part of the rules: > Third-party libraries for cryptographic primitives (erasure-coding, Bandersnatch, Ed25519), codecs (e.g. SCALE), and networking (e.g. QUIC) are acceptable.

2024-06-02 20:38 gav: FFI is fine.

2024-06-02 20:39 gav: Conformance tests are being worked on and we should be releasing the first batch in the next few weeks.

2024-06-03 06:55 tomusdrw: Hello! Is TypeScript cude enough for set C? Or rather: does it qualify as JS?

2024-06-03 07:01 tomusdrw: Also, if I calculate correctly, in the happiest case, the maximal number of teams getting the maximal prize for each milestone in every set is ~2-3 (10M - minus extra auditing/operational costs / 5 milestones / 4 sets / 100k prize). Hence I'm wondering is it public (maybe published already?) how many applications was there for each and every set?

2024-06-03 12:33 purpletentacle: Is there any additional information about networking specs/protocol? I could only find that it will be based on QUIC.. but nothing more than that..
This is not critical information for the first milestones.. but it can be useful to have in mind already. Any public information or hints about this? (edited)

2024-06-04 02:02 gav: Not really - expect QUIC with Ed25519 keys.

2024-06-04 02:03 gav: And direct connections to all other validators.

2024-06-04 02:03 gav: Networking specs probably won’t arrive for another 2-3 months.

2024-06-04 02:08 gav: 20240604T020818Z.jpg

2024-06-04 02:10 gav: Current thinking is that validators will be well connected with each other and then have a few extra proxy nodes which also are reasonably well connected. These then connect out to the public.

2024-06-04 03:36 xlchen: will the graypaper to cover RPC spec as well?

2024-06-04 04:15 gav: No (since it’s app specific and not related to the protocol).

2024-06-04 04:15 gav: But there should be a fellowship RFC.

2024-06-04 04:16 gav: It can be something implementor teams can discuss in the implementation conferences.

2024-06-04 04:17 gav: I expect to publish 0.2.0 of the paper today.

2024-06-04 04:17 gav: This should more or less finalize the DA system. (edited)

2024-06-04 04:18 gav: Safrole should also be final. And the trie spec. For all three we’ll aim to have test vectors in the coming days.

2024-06-04 04:19 gav: Block execution vectors should be ~6 weeks out.

2024-06-04 04:20 gav: Availability system test vectors probably about the same timeline. (edited)

2024-06-04 04:23 gav: Once we see some initial action on code from the first few teams then we’ll sort out a mini implementation conference/seminar/hackathon.

2024-06-04 05:23 pos.digitalghost: > <@gav:polkadot.io> And direct connections to all other validators. wonder how would this pan out in censored areas where data center IPs are blocked

2024-06-04 05:59 gav: The protocol does rely on high quality connections (400 Mbps sustained) between validators; if you’re running a validator node then you’d need to provide this level of connectivity.

2024-06-04 06:01 gav: For regular (“full”) nodes then you’d most likely connect to the validator proxies. For work-package authors, you’d want to connect to one of three guarantors of the core for the work package. Likely that would have a different protocol, and would not receive eg block updates over it.

2024-06-04 06:02 gav: All this networking stuff is the last thing to be finalized as it doesn’t affect the core protocol.

2024-06-04 09:36 tomusdrw: > <@tomusdrw:matrix.org> Hello! Is TypeScript cude enough for set C? Or rather: does it qualify as JS? ^^^ @gav any comments on this one?

2024-06-04 09:41 pos.digitalghost: guess so as it’ll have to convert to js anyways

2024-06-04 09:43 gav: > <@tomusdrw:matrix.org> Hello! Is TypeScript cude enough for set C? Or rather: does it qualify as JS? It qualifies as a major dialect of JS :)

2024-06-04 09:44 gav: > <@tomusdrw:matrix.org> Also, if I calculate correctly, in the happiest case, the maximal number of teams getting the maximal prize for each milestone in every set is ~2-3 (10M - minus extra auditing/operational costs / 5 milestones / 4 sets / 100k prize). > Hence I'm wondering is it public (maybe published already?) how many applications was there for each and every set? Auditing would be paid separately to the prize fund.

2024-06-04 09:45 gav: So it’s strictly a max of 500k DOT total (100k per milestone)

2024-06-04 09:46 gav: Gauging from interest thus far, I’d say it’s a reasonable expectation that teams delivering milestones pre-launch will be getting the full amount.

2024-06-04 09:49 gav: But yeah, theoretically if there’s more than 20 teams delivering viable, audited implementations across the language sets pre-1.0 spec then even those in the first wave could end up with less. I’d say it’s rather an unlikely eventuality. (edited)

2024-06-04 09:57 gav: If it’s an issue for teams then we could perhaps say that those teams who get to M1 first get prioritised for prizes if there is contention and they are at M4 or M5 at network launch. (edited)

2024-06-04 13:00 purpletentacle: > <@pos.digitalghost:matrix.org> > wonder how would this pan out in censored areas where data center IPs are blocked > @gav It would be nice if networking is ipv6 based.. this would simplify a lot the networking, nat, etc.. given that the spec will require at least 400Mbps.. most providers will be able to support ipv6 without complaining much...

2024-06-04 21:51 gav: > <@purpletentacle:matrix.org> @gav It would be nice if networking is ipv6 based.. this would simplify a lot the networking, nat, etc.. given that the spec will require at least 400Mbps.. most providers will be able to support ipv6 without complaining much... Yes indeed.

2024-06-07 01:01 aedigix: > <@gav:polkadot.io> Yes indeed. There should be support for ipv4 RPC endpoints unless RPC is outside of the scope of JAM

2024-06-07 04:01 gav: It’s essentially out of scope.

2024-06-07 04:03 gav: There might be an RFC. But JAM usage should rely on light-clients rather than full-node+RPC.

2024-06-10 20:37 purpletentacle: gav: In the case of some small inconsistencies in the graypaper, would you prefer to discuss them here, or a PR in the repo is better?

2024-06-11 02:30 gav: Best to discuss in the graypaper channel rather than here but if they’re obvious corrections then a PR is fine too.

2024-06-16 12:57 ltfschoen: it says here https://jam.web3.foundation/rules that "Clean-room implementation using the Graypaper and public implementor chat channels as the only resources.". it is my understanding that the Graypaper chat channel is https://matrix.to/#/#graypaper:polkadot.io where are the "public implementer" chat channels? and is there a reason why it doesn't include the Jam chat channel here https://matrix.to/#/#jam:polkadot.io? also, it says "Each team is only allowed to work on one implementation", but in the application form it says "What programming language(s) are you using and which language set are you applying for? e.g. "Rust, set B", so i don't understand why it's asking us "What programming language(s) are you using". is it just wanting to know if we'll be using multiple languages in the our single implementation (e.g. if we'll be doing an implementation in Rust and using FFI from Ruby and Python then we'd answer Rust, Ruby, Python, set B? i was actually going to try and work on multiple implementations using multiple language sets (e.g. Rust, Swift Ruby, Python, JS/TS) in parallel as a contingency incase i got stuck and couldn't get support with one of them. after submission of our initial application form, will be possible for us to later on change what programming language(s) and language set that we'll be using for the one implementation that could be eligible for the JAM prize?

2024-06-16 20:09 sourabhniyogi: For rule 7, "Any JAM-implementation code which is viewed before or during implementation must be declared", is the following acceptable for declaration:

In our implementation, beyond the Graypaper, we relied on the following for either FFIs or our learning of JAM:
 1. Koute's PVM Implementation: (no FFI)
    https://github.com/koute/polkavm
 2. Ring VRF: (FFI [link](...))
    https://github.com/w3f/ring-vrf
 3. Sassafras (no FFI)
    https://github.com/paritytech/polkadot-sdk/pull/1336/
    https://github.com/polkadot-fellows/RFCs/pull/26
 4. Polkadot-SDK: Grandpa (no FFI)
    https://github.com/paritytech/polkadot-sdk/...
 5. Polkadot-SDK: Erasure Coding (FFI [link](...))
    https://github.com/paritytech/polkadot-sdk/...

Are all of (1)-(5) declarations actually necessary? Do any of (1)-(5) break "clean room" expectations in some way? If the FFI vs non-FFI boundary is made clear and what should NOT be peeked at (even for learning purposes), it will spare implementers some grief (implementing something where FFI is acceptable or preferred, peeking at something which should not be peeked at). Can you make clear which components would ideally NOT use FFI and would ideally use FFIs? (edited)

2024-06-16 20:49 sourabhniyogi: For rule 15, "For code developed in private, commit hashes should be placed, in a timely fashion, ..." what does "in a timely fashion" mean, e.g if we commit via system.remark via https://assethub-polkadot.subscan.io/extrinsic/6489450-2 on June 16 and then again on July 16, Aug 16, Sept 16 (once a month) is that sufficient or is a higher frequency expected? (edited)

2024-06-17 03:07 ascriv: Will there be a public list of teams and their chosen programming language? I want to work on implementation but I also want not to spend a lot of time on one of the languages that has a seriously good team already on it

2024-06-17 03:07 ascriv: Could also be good for finding teams to join

2024-06-18 03:11 gav: The only thing which really matters is the code and the ability to demonstrate that it is a reasonably independent implementation of your own. (edited)

2024-06-18 03:12 gav: Application forms are totally irrelevant.

2024-06-18 03:13 gav: The point of restricting a prize-teams to a single implementation project is: (edited)

2024-06-18 03:14 gav: - to disincentivize the submission of additional Milestone 1/2 submissions over developing an impl towards Milestones 3-5. - to avoid getting multiple implementations with the same overall architecture and design (and bugs). (edited)

2024-06-18 03:18 gav: 1 compromises hygiene somewhat, 3 and 5 probably not since neither are compatible with GP. 2 and 4 are probably fine since they're not going to be defined in the GP. (edited)

2024-06-18 03:19 gav: Third-party libraries (FFI) for cryptographic primitives (erasure-coding, Bandersnatch, Ed25519), codecs (e.g. SCALE) and networking (e.g. QUIC) are acceptable. (edited)

2024-06-18 03:20 gav: Everything else should be properly implemented.

2024-06-18 03:21 gav: A proper Grandpa impl should be assumed to be a requirement of milestone 2 and above. This might be elevated to milestone 3, so treat it as the lowest priority feature. I haven't made this especially clear yet since it's not especially well-defined in the GP and I wouldn't expect impl teams to need it for some months to come yet. (edited)

2024-06-18 03:23 gav: > <@sourabhniyogi:matrix.org> For rule 15, "For code developed in private, commit hashes should be placed, in a timely fashion, ..." what does "in a timely fashion" mean, e.g if we commit via system.remark via > https://assethub-polkadot.subscan.io/extrinsic/6489450-2 on June 16 > and then again on July 16, Aug 16, Sept 16 (once a month) is that sufficient or is a higher frequency expected? Higher frequency

2024-06-18 03:24 gav: The point of this is to help _you_ prove that you wrote the code organically. (edited)

2024-06-18 03:25 gav: If you come to the Fellowship with one big commit of a working impl but the code design/arch looks similar to some other (maybe OSS) impl, then Fellowship are going to need further clear evidence that it's not just a port.

2024-06-18 03:26 gav: A Git history with credibly timestamped commit hashes are one way of helping with this.

2024-06-18 03:26 gav: If you don't provide this additional evidence then you run the risk of trouble with claiming the prize.

2024-06-18 03:27 gav: Teams which want to make their presence known can contact David at the W3F or post on here. (edited)

2024-06-18 03:28 gav: There is no requirement to pre-announce ones intention though. There is not a single prize per language: e.g. two teams writing an independent C++ impl can both claim the full prize in principle (clean room rules still apply!) (edited)

2024-06-18 03:29 gav: However, if you want to make it a bit more likely that you'll not be writing in the same language as some other team (for whatever reason), then consider selecting a less-common language. The language groupings are set with the intention of incentivising diverse impl languages. There's a single pot per grouping so that 20 teams all writing in C++ and Rust don't take the full pot. (edited)

2024-06-18 16:59 sourabhniyogi: For rule 24, "The repository must include a clear and permissive open-source license." are Business Source Licenses such as that of (1) Uniswap https://github.com/Uniswap/v3-core/blob/d8b1c635c275d2a9450bd6a78f3fa2484fef73eb/LICENSE (2) Arbitrum https://github.com/code-423n4/2024-05-arbitrum-foundation/blob/main/LICENSE.md (3) Aave https://github.com/aave/aave-v3-core/blob/master/LICENSE.md which mostly originate from https://mariadb.com/bsl11/ considered "clear and permissive open-source licenses"? It may be useful to have JAM teams get a head start over TRON/BSC/... type actors with a mechanism like this: https://github.com/Uniswap/v3-core/blob/d8b1c635c275d2a9450bd6a78f3fa2484fef73eb/LICENSE#L18-L21 (edited)

2024-06-18 17:05 gav: Could you make this question clearer/more specific? (edited)

2024-06-18 17:19 sourabhniyogi: Sure -- for Rule 24 "The repository must include a clear and permissive open-source license." is this license acceptable:

Business Source License 1.1

License text copyright (c) 2017 MariaDB Corporation Ab, All Rights Reserved.
"Business Source License" is a trademark of MariaDB Corporation Ab.

-----------------------------------------------------------------------------

Parameters

Licensor:             JAM Team X

Licensed Work:        JAM X  
The Licensed Work is (c) 2024 JAM Team X

Change Date:          2029-04-19

Change License:       GNU General Public License v2.0 or later

-----------------------------------------------------------------------------

Terms

The Licensor hereby grants you the right to copy, modify, create derivative works, redistribute, and make non-production use of the Licensed Work. The Licensor may make an Additional Use Grant, above, permitting limited
production use.

Effective on the Change Date, or the fourth anniversary of the first publicly available distribution of a specific version of the Licensed Work under this License, whichever comes first, the Licensor hereby grants you rights under the terms of the Change License, and the rights granted in the paragraph above terminate.

If your use of the Licensed Work does not comply with the requirements currently in effect as described in this License, you must purchase a commercial license from the Licensor, its affiliated entities, or authorized
resellers, or you must refrain from using the Licensed Work.

All copies of the original and modified Licensed Work, and derivative works of the Licensed Work, are subject to this License. This License applies separately for each version of the Licensed Work and the Change Date may vary for each version of the Licensed Work released by Licensor.

You must conspicuously display this License on each original or modified copy of the Licensed Work. If you receive the Licensed Work in original or modified form from a third party, the terms and conditions set forth in this License apply to your use of that work.

Any use of the Licensed Work in violation of this License will automatically terminate your rights under this License for the current and all other versions of the Licensed Work.

This License does not grant you any right in any trademark or logo of Licensor or its affiliates (provided that you may use a trademark or logo of Licensor as expressly required by this License).

TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS, EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND TITLE.

MariaDB hereby grants you permission to use this License’s text to license your works, and to refer to it using the trademark "Business Source License", as long as you comply with the Covenants of Licensor below.

-----------------------------------------------------------------------------

Covenants of Licensor

In consideration of the right to use this License’s text and the "Business Source License" name and trademark, Licensor covenants to MariaDB, and to all other recipients of the licensed work to be provided by Licensor:

1. To specify as the Change License the GPL Version 2.0 or any later version, or a license that is compatible with GPL Version 2.0 or a later version, where "compatible" means that software provided under the Change License can be included in a program with software provided under GPL Version 2.0 or a later version. Licensor may specify additional Change Licenses without limitation.

2. To either: (a) specify an additional grant of rights to use that does not impose any additional restriction on the right granted in this License, as the Additional Use Grant; or (b) insert the text "None".

3. To specify a Change Date.

4. Not to modify this License in any other way.

-----------------------------------------------------------------------------

Notice

The Business Source License (this document, or the "License") is not an Open Source license. However, the Licensed Work will eventually be made available under an Open Source License, as stated in this License.

If not, is there a specific Change Date modification that could be acceptable?

2024-06-18 17:21 gav: Without consulting a lawyer, no and no. (edited)

2024-06-18 17:23 gav: The point is to ensure that any implementations are the property of the coders who deign to maintain them. The licenses should reflect that. (edited)

2024-06-18 17:23 gav: Licenses which allow for individuals, organisations or other economic entities to retain a privilege over a code base regardless of others’ contributions do not fulfil this goal. (edited)

2024-06-19 05:49 hitchhooker: https://i.ibb.co/z2TNfrD/toaster.png tried to draw toaster based on images and 2049 specs. looks like we around 128kw in total?

2024-06-19 12:16 sourabhniyogi: > <@gav:polkadot.io> Safrole should also be final. And the trie spec. For all three we’ll aim to have test vectors in the coming days. Are the test vectors for { Safrole, trie, DA system (?) } published already (if so, where?) or to be published still (if so, is there an ETA)?

2024-06-19 12:25 tomusdrw:

2024-06-19 13:23 gav: > <@sourabhniyogi:matrix.org> Are the test vectors for { Safrole, trie, DA system (?) } published already (if so, where?) or to be published still (if so, is there an ETA)? Not yet. Will notify here as soon as they’re compiled.

2024-06-20 17:55 yikesawjeez: hey; is there a canonical resource for what all a jam client would entail? Closest thing I can find is the RFC, but i have a team working on a substrate solochain that's more-or-less what's being described in the greypaper; would love to find some more info

2024-06-21 09:57 gav: The Graypaper covers all of the main protocol elements. There’s a notable lack of spec for a full networking protocol, which doesn’t make much sense to do until the first team gets basic block importing working (Milestone 1). (edited)

2024-06-21 09:58 gav: “More or less” isn’t especially useful and it cannot be implemented utilising Substrate as there are several design decisions in Substrate which would preclude specifically this protocol. (edited)

2024-06-21 15:43 jimboj21: Going off the previous question, is it imagined that certain components of JAM will use parts of substrate, or will everything be written from scratch? A specific example I am thinking of is grandpa. Is it imagined that the current implementation will be used (for one of the impls) or that it will be rebuilt?

2024-06-21 17:59 sergei_astapov: Hey everyone, Three passionate PBA 5 alumni have chosen to build a JAM client using Scala 3 (vanilla Scala). If you're familiar with Scala and interested, feel free to DM me without hesitation! Thanks!

2024-06-22 04:16 gav: Mostly from scratch; Substrate specifically won't be helpful. (edited)

2024-06-22 04:16 gav: Substrate is a relatively opinionated blockchain framework. It is also software, not a protocol. (edited)

2024-06-22 04:16 gav: Jam is a protocol. That essentially precludes it from being based on Substrate, which is software.

2024-06-22 04:19 gav: Grandpa is also protocol-first, so pre-existing implementations can likely be leveraged provided they are compatible with the Grandpa spec paper and the (yet to be formalised) GP-integration spec.

2024-06-22 04:24 gav: For the full payout, we'll almost certainly require a clean-room implementation of Grandpa though. Basing all node impls on a single Grandpa impl could compromise the network in case a bug were found in it. In general, the only exceptions we can make to this rule are where the logic is extremely well-specified and understood outside of the Jam/Polkadot community, audited, and/or in heavier or more economically significant use outside of Polkadot. For this reason we can make exceptions for some of the underlying crypto/networking/math primitives which are not especially specific to Jam/Polkadot, such as EC, Ed25519, BLS, QUIC, Bandersnatch. (edited)

2024-06-22 04:28 gav: As I mention above though, implementing Grandpa should not be considered an especially high-priority job. At present the definition of milestone 2 would include it, but this might very well change to milestone 3 in due course.

2024-06-22 04:58 tomusdrw: > <@gav:polkadot.io> - to disincentivize the submission of additional Milestone 1/2 submissions over developing an impl towards Milestones 3-5. > - to avoid getting multiple implementations with the same overall architecture and design (and bugs). Given the objective of the "one team = one implementation" rule is to have teams focus on milestones 3-5 is it allowed to change the implementation language at some point? I'm thinking specifically about a situation, where say, the first two milestones are completed in a less performant language (e.g. lang set C) and then the team realises there is almost no way of completing 3-5 and switch to something faster. Obviously the prize for Milestones 1-2 wouldn't be claimed twice, the second implementation would only claim 3-5. Would that be acceptable?

2024-06-22 05:00 gav: Totally.

2024-06-22 05:01 gav: I'd also leave open the degree to which M1 and M2 could be claimable for the second language.

2024-06-22 05:03 gav: If the two languages are different to the point that one could reasonably claim that it would be difficult for bugs and design flaws to cross the linguistic divide (e.g. functional vs procedural), then the Fellows (or failing that, W3F) could reasonably decide to award some or all of the prize anyway. (edited)

2024-06-22 05:04 gav: Ultimately, the point of the prize is to build resilience through decentralisation and diversity. The rules are (hopefully) relatively conservative in preventing loopholes which might allow teams to claim the prize without contributing to the network's resilience. But where that conservatism might be judged by the Fellowship to be counter-productive, then it might reasonably choose to award prizes regardless. (edited)

2024-06-22 05:07 gav: Obviously if a team implements in C and then "reimplements" in a C-dominated "C++", then this would not be claimable.

2024-06-22 05:08 gav: But if they implement in Haskell and implement again in a very idiomatic C++, both _might_ be claimable. All prize decisions are made at the discretion of the (non-team) Fellows. (edited)

2024-06-22 05:13 gav: Languages from different language sets and which do not share syntactic DNA would be a sensible lower-bar (required but not sufficient) for multiple prizes. (edited)

2024-06-22 19:59 purpletentacle: I think it would be unfair to the ones that get to M3-4.... to see teams that just concentrate on doing many M1s.. The biggest risk I see, is that performance requirements are not very clear.. and very likely Rust will set the bar for every other language... having some guidelines on what performance (%) is acceptable below the top implementation would be useful (edited)

2024-06-22 21:17 xlchen: the bottleneck is most likely to be pvm and for production performance we most likely need to translate it to native code and run it. in that case, the language will only impact the translation phase, which should be a relatively simple thing

2024-06-22 21:17 xlchen: so the implementation language shouldn’t matter too much

2024-06-22 21:20 xlchen: but of course we can’t say for sure without an implementation

2024-06-23 03:05 gav: > <@purpletentacle:matrix.org> I think it would be unfair to the ones that get to M3-4.... to see teams that just concentrate on doing many M1s.. > The biggest risk I see, is that performance requirements are not very clear.. and very likely Rust will set the bar for every other language... > having some guidelines on what performance (%) is acceptable below the top implementation would be useful M5 should be a pretty easy milestone to achieve after M4 for any well-engineered codebase, so that helps mitigate against bias towards "reimplementing what you know". In fact, roughly half of the actual logic is in M1 and roughly the other half in M2. M3, M4 and M5 are essentially "just" optimisations and security, so teams who repeatedly reimplemented M1 would probably be losing out. But point taken, and this is why the rules are as they are and it cannot be taken for granted that the same milestones by the same people will receive multiple rewards.

2024-06-23 03:06 gav: > having some guidelines on what performance (%) is acceptable below the top implementation would be useful One quite likely possibility is that likely performance requirements will be set at the lower of 80% of the most performant implementation and the third-most performant implementation. It will almost certainly need to be relative to the performance of the actual implementations. Another possibility would be to be just 90% of the 3rd most performant impl. (edited)

2024-06-23 03:11 gav: Indeed, performance bottlenecks will almost certainly be database (which is designed to be held entirely in-memory anyway), PVM (designed for recompilation) and crypto/EC (designed to allow SIMD and aggregation, and for which FFI/external libraries are acceptable). Actual on-chain business logic is pretty minimal. (edited)

2024-06-23 03:17 gav: It would be up to the team to figure out how to make the performance criteria happen. For example, a JVM-based language would probably have to do the same trick as native-built language and recompile PVM to native machine language and execute natively somehow, rather than doing the naive, fully-compatible thing and recompiling into JVM.

2024-06-23 06:46 sergei_astapov: Hi all, In graypaper pg 8,9

(18) β′ ≺ (H,EG,β†,C)
(28) (..., C) ≺ (EA,ρ′,δ†,χ,ι,φ)

I'm unable to determine what 'C' really stands for. Please correct me if I'm mistaken, but it seems there is no definition for C in GP. Thank you! (edited)

2024-06-23 06:47 gav: It’s the beefy commitment set.

2024-06-23 06:47 gav: It should be mentioned in the definitions section.

2024-06-23 06:48 gav: It’s defined at the end of the accumulation definitions.

2024-06-23 06:48 gav: Section 14 iirc.

2024-06-23 15:13 sourabhniyogi: For our travel budget/life, I would very much like to see the implementer's hackathon/get-together at Devcon 7 https://devcon.org/en/ in November \[like 2 days before then, the 4 days during, and then maybe 2 days after with a bunch of JAM how-is-this-really-working nitty gritty\] -- is this possible to get on a calendar? If you mean to have a get-together earlier than that, can you let us know so we can plan our travel life/budget accordingly? I'd like to believe around 4+ teams can get Milestone 1 in "local tests passed!" shape before this get-together and have like "hey, lets see if we can network 4 different implementations together for real" sort of hacking happening there, and get ssh-login concrete about whatever private access to standard hardware / JAM TOASTER will mean. Then we can all feel pretty good about getting Milestone 2 by Q1 2025 and finishing Milestone 1 by the end of the year. What do you think? (edited)

2024-06-24 02:39 gav: This seems like a good target timeline.

2024-06-24 04:11 xlchen: I created some diagrams, will love a review https://hackmd.io/0gSmXyElT2iKawymS5jJKw?view

2024-06-24 04:30 sourabhniyogi: This is super helpful (and keeps us all grounded, thank you!). Did we miss a lecture on the Asset Hub <=> JAM connection, or is this your idea on how to keep us grounded? I was wondering what happened to this sketch/intuition: https://github.com/polkadot-fellows/RFCs/blob/gav-coreplay/text/coreplay.md#scaling-a-common-use-case

2024-06-24 05:32 gav: > <@xlchen:matrix.org> I created some diagrams, will love a review https://hackmd.io/0gSmXyElT2iKawymS5jJKw?view Looks pretty accurate.

2024-06-24 05:33 gav: I’d generally try to avoid RPC in Jam completely, and just try to push the use of one or more Smoldot-style light clients.

2024-06-24 05:34 gav: At present there’s no prize allocated to specifically light clients, but I do expect to incentivise their creation in due course.

2024-06-24 05:41 xlchen: Users don't really need to access the JAM chain anyway so I can image we only need some RPC for operators/developers

2024-06-24 05:50 gav: Yes if it’s just about providing an FFI and if they are generally wanting to run a full node then fair enough. But a full node is not trivial to run - every 6-second block is projected to have ~3.5 seconds of PVM computing in it plus a large amount of signature verification and memory access for the DB. The blocks are big: maybe 17 MB every 6 seconds. For work package building nodes I’m not sure they’d necessarily want to be running a full node. (edited)

2024-06-24 05:52 gav: Some will need the raw state read throughout. Many won’t: parachain collators for example probably only need to read 1 or 2 KB per block which could easily be managed by a light client and really doesn’t make sense to run a full node over. (edited)

2024-06-24 05:53 gav: And we really don’t want them connecting to some random shared RPC for it. (edited)

2024-06-24 06:00 xlchen: we could have a flag to disable most of the signature verifications and skip some other checks and only rely on grandpa/beefy instead and it will be lightweight enough for many use cases? but yeah I can totally see that majority of the nodes are light node and not much need to have rpc node/full node

2024-06-24 06:02 gav: Well you’d still need to be running 3.5s of PVM, 1M db iops and downloading 17MB every 6 seconds.

2024-06-24 06:02 gav: Even if you skipped all the signature checks.

2024-06-24 06:06 xlchen: right. so we still need a proper light client and some protocol for the light client to communicate with full node

2024-06-24 06:07 gav: Indeed; thankfully much of the work is already done with Smoldot.

2024-06-24 10:57 purpletentacle: > 13. A clear Git history and public, credibly timestamped commits are necessary in order to help evidence organic code development by the team/s. > 14. Timestamps may be by virtue of pushing code to GitHub on a timely basis. > 15. For code developed in private, commit hashes should be placed, in a timely fashion, on a major public blockchain and readily visible on a block explorer. Just to avoid any kind of future misunderstanding about this. IF the project is done in GitHub, the requirement for pushing hashes into a blockchain is not necessary? Or for that to be fine, the github project needs to be made public and posted here? Or actually to be on the safe side, every project is expected to publish commit hashes to a blockchain every N days? (edited)

2024-06-24 13:17 gav: Fellowship will be looking for evidence that the team coded the software themselves from the GP. That it’s not a port or AI adjusted version of some other team’s software. To this extent it’s important for the teams to demonstrate an organic development process. Regular, sensibly-sized commits are one way of doing this and we assume GitHub is reasonably corruption-resistant (git alone obviously isn’t). But if you don’t want to use GitHub then make sure you credibly timestamp the commits elsewhere such as on a public blockchain. (edited)

2024-06-24 13:18 gav: If you dump a codebase at the doorstep of the Fellowship with no way of proving that you actually coded it yourself then you run a risk of getting no prize.

2024-06-24 13:18 gav: This becomes especially important in the age of generative AI.

2024-06-24 19:43 danicuki: I was wondering what programming language could result in the best performing implementation of JAM. If JAM is a CPU bound system, probably C, C++ or Rust could give me the best performance If JAM is a I/O bound system, a language with more concurrent capabilities would be better. Which one would be the case? I am an enthusiast of Elixir, a very powerful language for concurring systems. I feel it could have a very elegant and yet performatic version of JAM

2024-06-25 02:52 gav: Difficult to say at this stage though I can't see how there'd be any way around having PVM be recompiled into native and good optimised algorithms for EC and crypto, however these are very discrete components and with the latter two could reasonably be imported in. (edited)

2024-06-25 06:40 gav: @room The first test vectors have been merged into github.com/w3f/jamtestvectors (edited)

2024-06-25 06:41 gav: So far it's Safrole's STF with a couple of PRs open for the Merkle trie and PVM. (edited)

2024-06-25 06:41 gav: We'll work on getting vectors for the rest of the STF in the coming weeks, as well as the baseline crypto and EC. (edited)

2024-06-25 11:21 emielvanderhoek: I am looking forward to the Timeslot’s STF vectors. 😆

2024-06-27 04:56 xlchen: Screenshot 2024-06-27 at 4.56.16 PM.png

2024-06-27 04:56 xlchen: Screenshot 2024-06-27 at 4.56.31 PM.png

2024-06-27 04:56 xlchen: the order of the keys is different between GP and the test vector

2024-06-27 04:57 xlchen: https://github.com/w3f/jamtestvectors/blob/fa8699f94b2769cee5d05472072e6a5f839a57ed/safrole/safrole.asn#L35

2024-06-27 05:10 xlchen: looks like the safrole test vectors are encoded using JSON instead of the scale codec described in GP. i.e. a validator data is

{
                "ed25519": "0x837ce344bc9defceb0d7de7e9e9925096768b7adb4dad932e532eb6551e0ea02",
                "bandersnatch": "0xf16e5352840afb47e206b5c89f560f2611835855cf2e6ebad1acc9520a72591d",
                "bls": "0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
                "metadata": "0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
            },

instead of bunch bytes. Will it be better to offer a variant that's using the raw bytes instead of JSON?

2024-06-27 05:10 xlchen: asking because otherwise 1) we need to implement a JSON serialization function for the types 2) need to ensure the names matching to the tests vector

2024-06-27 05:14 xlchen: and we cannot use the test vectors to test the serialization part

2024-06-27 08:11 gav: > <@xlchen:matrix.org> the order of the keys is different between GP and the test vector Yes indeed @davxy:matrix.orgcould you fix this?

2024-06-27 08:13 gav: > <@xlchen:matrix.org> looks like the safrole test vectors are encoded using JSON instead of the scale codec described in GP. i.e. a validator data is >

> {
>                 "ed25519": "0x837ce344bc9defceb0d7de7e9e9925096768b7adb4dad932e532eb6551e0ea02",
>                 "bandersnatch": "0xf16e5352840afb47e206b5c89f560f2611835855cf2e6ebad1acc9520a72591d",
>                 "bls": "0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
>                 "metadata": "0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
>             },
> 

> instead of bunch bytes. Will it be better to offer a variant that's using the raw bytes instead of JSON? Probably yes. I’m not against having smaller data items separately encoded (as it adds to the readability of the vectors) but in this case our primary definition is the blob, and the individual keys are just equivalence derivatives.

2024-06-27 08:14 gav: @davxy:matrix.orgcould you fix this also?

2024-06-27 15:51 davxy: Sure. I'll put the fix as part of an upcoming PR where I'll propose the vectors in two flavors: tiny (vals: 6, epoch len: 12) and prod (vals: 1023, epoch len: 600). PR very soon 🙂

2024-06-27 15:55 davxy: Ah. As you may have noticed, both metadata and bls key are placeholders. Not a big deal BTW, as they are not influencing Safole's STF

2024-06-28 06:47 kianenigma: I am trying to make sure my understanding of the semi-coherence property of JAM is correct. My understanding of this property is that parts of the system (i.e. *services*) that need to talk a lot with one another should be close to one another to do so without needing to deal with asynchrony. The most ideal system would then be one in which services can talk synchronously to the services that happen to execute in the same core in that time-slot, and asynchronous with those that are not. That being said, I believe I heard in the Singapore Graypaper lecture that the interfaces for services to communicate with one another is always asynchronous. If correct, we can then say that services, as the lowest level programming primitive of JAM don't enable synchronous comparability yet, but synchronous composability as described in a semi-coherent system can be built on top of them. This is primarily possible because of the core-model, the possibility of flexible scheduling of services in cores, and the PVM's ability to be readily paused and resumed. One example of such system is CorePlay. Is that a correct description of where JAM service stand in terms of synchronous composability between one another?

2024-06-28 06:50 emielvanderhoek: Have recordings of the university lectures been made available, or if not will they be made available?

2024-06-28 07:12 gav: > <@kianenigma:parity.io> I am trying to make sure my understanding of the semi-coherence property of JAM is correct. > > My understanding of this property is that parts of the system (i.e. *services*) that need to talk a lot with one another should be close to one another to do so without needing to deal with asynchrony. The most ideal system would then be one in which services can talk synchronously to the services that happen to execute in the same core in that time-slot, and asynchronous with those that are not. > > That being said, I believe I heard in the Singapore Graypaper lecture that the interfaces for services to communicate with one another is always asynchronous. > > If correct, we can then say that services, as the lowest level programming primitive of JAM don't enable synchronous comparability yet, but synchronous composability as described in a semi-coherent system can be built on top of them. > > This is primarily possible because of the core-model, the possibility of flexible scheduling of services in cores, and the PVM's ability to be readily paused and resumed. One example of such system is CorePlay. > > Is that a correct description of where JAM service stand in terms of synchronous composability between one another? Basically yes. The fact that the cores are stateless and that there's a mostly-coherent shared data lake and a fully-coherent reactor state means that there's no structural decoherence in the same way as with persistently sharded systems like the Polkadot Relay and the present direction of Eth2.

2024-06-28 07:12 gav: You can still build decoherent systems on this (the basic pre-Accords Parachains service will be an example of this). But CorePlay will (hopefully) use this model to create a coherent, performant and highly scalable platform on top.

2024-06-30 15:35 purpletentacle: > looks like the safrole test vectors are encoded using JSON instead of the scale codec described in GP I think it makes sense to separate concerns correctly. Otherwise, we will end up almost having something like integration tests... where the only test vector for M1 ends up being a complete block.. Moreover, some of us already implemented the JSON decoder.. because that's the way it was delivered ! ;) Anyway, this is not a big deal at the moment.. but it would be good to have some stability on the test vectors anyway to avoid too much rework Having said that... I would prefer if we can keep the JSON ones.. add some for encoding/decoding.. and later on, have complete ones for actual headers and blocks (edited)

2024-06-30 16:26 purpletentacle: davxy I just saw that you have a PR with both json and scale! that's amazing 🚀

2024-06-30 18:03 sourabhniyogi: I have been bothered by the following line of questioning for the last couple of months: - RFC 32 set a "Minimal Relay Chain" architecture for Polkadot where the functions of relay chain would be "split off" into new system chains: Asset Hub, People, Staking, Core Time, and so forth. The idea was that the Relay Chain REQUIRED maximal resources for validating parachains, specifically on "accumulate" computations. In particular, the relay chain should not have anything like contracts-pallets because that would compromise on "accumulate" scalability \[to validate parachains\]. Here is your view circa October 2023: https://github.com/polkadot-fellows/RFCs/pull/32#discussion\_r1347502727 - But then you put together JAM in the last year (and, this contracts-pallet based on PVM appears to be coming together nicely!). JAM will have as its first really big JAM Service a Parachain Validation Service (run in-core and on-chain in refine vs accumulate run in PVM byte code), validating system chains and parachains with work items in work packages, judgements, and so on. - Then, as Polkadot Fellows follow through on RFC 32 AND the JAM Service, there will be, in the limit, nothing left for the old relay chain to do! It leaves a black hole as baby JAM takes over. - Question 1: Instead of leaving the relay chain with nothing left to do, why not revert RFC 32's "splitting off" of the relay chain components and just leave some (or all) of it on the relay chain? - Question 2: In particular, in addition to contracts-pallet being on Asset Hub, can we revisit the old relay chain getting contracts-pallet now that the JAM Parachain Validation Service model is clear? What are the blockers other than the big one \[of how parachain validation will "switch" from Polkadot Relay Chain to the JAM Parachain Validation Service\]? - Question 3: If you believe the above is possible, then with a functioning JAM Parachain Validation Service, then what is presently called the "Polkadot relay chain" could become a (well connected) parachain like all others, complete with smart contracts. Problems like CEX integration to Asset Hub ... disappear, UX improvements to integrate disparate parachains are pushed off to another day while people figure out if some of these "split off" decisions from RFC 32 are better solved within a JAM Service architecture or as a parachain ... or as a smart contract on Polkadot (no longer validating anything) ... or elsewhere. Can we revert RFC 32 now that JAM is well architected in this way? If not, why not? (edited)

2024-06-30 23:14 danicuki: I created a YouTube Playlist with the main videos I found about JAM. Open to recommendations of videos to include. https://www.youtube.com/playlist?list=PLjGN96vX3BTKjLfBd5LimBuDuch-QfKPy

2024-07-01 08:03 gav: JAM can be considered a follow-on protocol, or upgrade, to “Relay”. (edited)

2024-07-01 08:04 gav: Its aim is to be fully backwards compatible. With JAM deployed, the Relay chain would be redundant and have no native currency or economy. Nothing stops validators from hard-forking and continuing to validate on the defunct chain, but they couldn't be paid unless they also forked DOT. And I'm not sure why you'd think this a sensible idea. (edited)

2024-07-01 08:07 gav: JAM is in line with the original proposal of moving extraneous functionality off the central, parachain-STF security chain (currently the Relay chain, to become the JAM chain). Indeed JAM requires this to happen as it cannot accept process transactions. (edited)

2024-07-01 08:08 gav: Polkadot Relay couldn’t become a “well-connected parachain” for exactly the same reason that hierarchical parachains are not viable right now. (edited)

2024-07-02 10:05 darkartistry:

2024-07-02 14:23 danicuki: Are the PBA JAM lectures already available somewhere?

2024-07-02 14:25 rtti-5220: danicuki: the recordings are a collab between Key Pictures and The Kus, and after post-production I am sure they will be published for everyone.

2024-07-02 14:25 rtti-5220: but not yet.

2024-07-04 02:02 xlchen: For the Safrole test vectors, it includes an error code but the value are unspecified and the README said we should ignore those values. This is a bit annoying as we need to do extra work to patch the test vectors to ignore the error code. Can we just exclude it from the test vectors? Or make it always 0. I just want to do a simple equality check

2024-07-04 02:04 xlchen: Also the test vectors repo should come with a test runner as well. As a fellowship member, how can I verify an implementation actually runs the test vectors correctly? We won't have such issue if it includes a test runner and if the tests passed for an implementation, we know for sure it actually passed.

2024-07-04 02:07 xlchen: and if we have a buildin test runner, then it can do the error code handling and it wouldn't be a concern to teams

2024-07-04 09:22 purpletentacle: hi Bryan, we are actually using the error codes and find them useful :( about the test runner... would that involve defining APIs for tests? It sounds that it would introduce architectural constraints to implementations at a very early stage. I think these kind of tests should be applicable at an integration level much later.. At the moment, we just add jamvectors as a submodule in our code and treat them as external data for simple unit tests..

2024-07-04 09:24 xlchen: how are you using the error codes? are the meanings defined somewhere?

2024-07-04 09:25 xlchen: I am not sure what kind of constraints will the API introduces. The most simple way will be require some CLI interface and use stdin/stdout to pipe test cases and results

2024-07-04 09:26 xlchen: all those tests are pure functional

2024-07-04 09:27 purpletentacle: There is a very small amount of tests... so it is easy to link the error code to the failure mode described in the README

- [publish-tickets-no-mark-3](publish-tickets-no-mark-3.json)
  - Fail: Re-submit tickets from authority 0.
- [publish-tickets-no-mark-4](publish-tickets-no-mark-4.json)
  - Fail: Submit tickets in bad order.
- [publish-tickets-no-mark-5](publish-tickets-no-mark-5.json)
  - Fail: Submit tickets with bad ring proof.
- [publish-tickets-no-mark-7](publish-tickets-no-mark-7.json)
  - Fail: Submit a ticket while in epoch's tail.

2024-07-04 09:28 purpletentacle: and we can match our own error code to that in the test.. at least it is not FAIL = true/false

2024-07-04 09:28 purpletentacle: anyway, different teams can try their own approaches.. of course! that's the beauty of having "only" the graypaper as a guide.... (edited)

2024-07-04 09:29 xlchen: in that case we should define them. as someone with C++ experience, I am scared of UB

2024-07-04 09:31 purpletentacle: well.. I don't think those have any relation to our code.. or any team's code for that matter.. those are 100% external data.. completely unrelated to implementations. worst case, they would affect the code in your tests. If those were to be defined.. they should be defined in the GP, right? which I find unnecessary (edited)

2024-07-04 09:32 xlchen: they can be defined in asn.1 file. there are other util types defined there as well

2024-07-04 09:33 purpletentacle: ok, if by definition you mean the test vectors ASN.1 and restrict that to only the test vectors.. welcome! I thought you wanted this defined and standardized for all implementations at the GP level. (edited)

2024-07-04 09:36 xlchen: yeah I just want it to be written down explicitly somewhere to have some stability guarantee and reduce guess work that we need to do

2024-07-04 09:36 purpletentacle: about external testing via stdio/stdout.. sounds good! happy to help with it if there is anything to do... actually having that as a github action would be actually cool... so we can all integrate it in our CI that way we can keep out own unit tests.. but also everyone can provide a "test cli" to run the "jam vectors action" .. (edited)

2024-07-04 09:47 gav: Error codes are not going to be defined in GP - way too implementation-specific. They're included in the test vectors as an aid for development only; for the purposes of determining conformance they can be collapsed into just "invalid"; implementations are not expected to be able to match the error code. (edited)

2024-07-04 09:48 gav: davxy: perhaps we should prefix the error-code field with an "\_" or something else to make clear it's not related to conformance? the only thing which should be officially there is that the posterior(s) are undefined due to invalid priors/inputs. (edited)

2024-07-04 10:25 davxy: Well for sure having error codes as an aid of "what went wrong" is useful. But I agree that it is an implementation detail and it must not go in the GP. So IMHO a good compromise is to: 1. (As Juan suggested) define the error codes only in the ASN.1 (as a hint to the implementer of why the test fails) 2. (As Gav suggested) make more explicit that the error code field is not related to conformance. Unfortunately _ can't be used in standard ASN.1 LOL. But I can rename it to something like application-specific-errno (with a big warning that this is not specified in the GP). (edited)

2024-07-09 00:13 danicuki: > <@gav:polkadot.io> @room The first test vectors have been merged into github.com/w3f/jamtestvectors A curiosity: how do you create these test vectors? And how can we be sure or prove they are correct?

2024-07-09 06:22 gav: They’re created with prototype implementations of the GP. If you think they’re incorrect, then raise an issue in the repo.

2024-07-09 06:24 gav: The GP is the canonical definition of JAM. At this point the test vectors are provided as a development aid to teams.

2024-07-09 06:24 gav: They will eventually mutate into a series of conformance tests for milestones 1 and 2.

2024-07-09 06:26 gav: You can prove (in)correctness in the usual way. The test vectors are formal and the GP is formal - show that one does (not) follow from the other.

2024-07-09 21:17 kianenigma: I have been working on a blog post explaining JAM in short form and super high level, sharing it here to gather feedback. I have yet to improve the graphics and enhance it with animations, but the content is more or less there. I hope it can also help implementors getting just started. I have done my best to explain the content ground-up: each section building on top of the previous one(s), and not making too many unstated assumptions. I particularly hope to receive feedback on this, and how the story of Polkadot 1 -> 2 -> JAM can be better explained. https://blog.kianenigma.nl/posts/tech/demystifying-jam/

2024-07-10 12:31 darkartistry: Hi, I think for the part of coherent or non-coherent systems, based on the decision of a Timing Model (Async/Sync) can be better explain with a table break down of the combinations of the Timing Model to the Failure Model and Channel Model. For technical readers. Good read. :) Could also add some information that it is a 5-10 min read. Cause the page looked really long at the start. (edited)

2024-07-10 19:18 sourabhniyogi: Strongly Web3 talk: https://x.com/thewhiterabbitM/status/1811009553565114766

2024-07-12 17:44 dakkk: Hi, I have few question about the JAM Prize Competition: 1. Can I apply as an individual? 2. After I submit the Apply form, should I receive any feedback from the foundation? 3. How can I know that I'm the only one working on a specific language? 4. How milestones works? After my software reaches a milestone what I'll have to do?

2024-07-12 18:31 gav: 1. The application is unimportant as far as the prize is concerned. The prize is for software. There’s no restrictions on the team behind it. 2. No idea about this form. As I say, it’s unimportant for the prize. 3. You can’t. But it doesn’t matter because teams need not implement in a unique language. 4. Just publish the milestone and notify here. The Polkadot Fellows will take a look.

2024-07-14 05:09 proxy720: Hi all! I have a question related to the one in the one above: - Since I'm very good with niche functional languages, I'm considering entering the competition with Haskell, Elixir, OCaml, F# or Scala. - Is it preferable to aim for the language that is more rare (the first 3), has some specific advantage over the others (Elixir)? - There is a restriction that a team can only enter with a single language, but can one person be part of multiple teams?

2024-07-14 05:14 proxy720: - Also, I'm considering implementing only a PVM interpreter/evaluator in Coq, which would allow formal verifications for pvm programs (smart contracts). Is there a prize for something like this?

2024-07-14 07:05 gav: Rarer languages are probably going to be in the less-likely language sets (C and especially D) and thus more likely to have the full prize pot available by the time of Jam’s launch.

2024-07-14 07:06 gav: Implementations are assumed to be clean room. If someone works on two different ones then obviously that’s no longer the case.

2024-07-14 07:09 gav: Basically all members of the team must adhere to the rules as if they were the only one.

2024-07-14 07:09 gav: > <@proxy720:matrix.org> - Also, I'm considering implementing only a PVM interpreter/evaluator in Coq, which would allow formal verifications for pvm programs (smart contracts). Is there a prize for something like this? No

2024-07-15 08:55 masapr: It's hard for me to estimate the effort to get to milestone 1. to get a better feeling for this: @gav if you had to implement milestone 1 by yourself in Rust, can you tell us how many hours you would expect to have on this?

2024-07-15 08:57 gav: Lol.

2024-07-15 08:58 gav: I really wouldn't base any of your projections on my productivity.

2024-07-15 09:16 masapr: Well, it's just a big risk to put a team on an unknown amount of work with unknown payment. If I know your estimate, I can draw my own conclusions on how that relates to our own productivity. ... Of course, we would have quite some additional effort understanding the graypaper in detail. But it would give some indication on the actual implementation effort. I understand, if you don't want to give a number. Just thought I'd ask. It would help me to know, if I should go further with this or stop right here ...

2024-07-15 09:18 gav: The spec is published and aims to be unambiguous. For milestone 1, I think it's pretty much there, too. (edited)

2024-07-15 09:18 gav: I see little risk beyond your own expertise.

2024-07-15 09:19 gav: I suggest you look at the JAM Prize as a *prize*, not a corporate SLA.

2024-07-15 09:25 gav: For what it's worth, while pre-existing companies are 100% welcome to code up JAM and claim the prize, the structure of the prize (along with Fellowship membership and support structures) is aimed more at individuals and small unincorporated teams: Those who believe themselves capable of bringing the Graypaper to life and don't mind taking a risk on their time and talent. If you're looking for a low risk way of getting a monthly income on some blockchain engineering, it may not be the most appropriate opportunity. (edited)

2024-07-15 09:34 masapr: Fair enough. Either way, it is an investment. Also for individuals. So, it does make sense to calculate an estimate of the costs. Anyone has to do that, really. Thanks for the feedback, though. It does give me an idea.

2024-07-15 09:35 xlchen: if you just want a number, I will say 3 to 6 months

2024-07-15 09:36 masapr: > <@xlchen:matrix.org> if you just want a number, I will say 3 to 6 months for 1 person?

2024-07-15 09:38 xlchen: yes. M1 is relatively easy when all the test vectors are out (which currently only Safrole is released and few others are WIP) as you can use the test cases to help understand GP

2024-07-15 09:38 xlchen: and it is pure functional without performance requirement

2024-07-15 09:39 masapr: Bryan Chen | Laminar & Acala: Thank you :-)

2024-07-15 10:25 gav: > <@masapr:matrix.org> Fair enough. Either way, it is an investment. Also for individuals. So, it does make sense to calculate an estimate of the costs. Anyone has to do that, really. Thanks for the feedback, though. It does give me an idea. I'd disagree wholeheartedly with any assertion that established businesses are anywhere near as capable of the agility, insight or risk-propensity as individuals.

2024-07-15 10:25 gav: But it's rather OT.

2024-07-15 10:26 gav: > <@masapr:matrix.org> Bryan Chen | Laminar & Acala: Thank you :-) Lol. One person who should be no more capable of reading and assessing the GP that you or your team says casually "3 to 6 months" and somehow you think it's helpful?

2024-07-15 10:27 gav: Perhaps it might be sensible to read and understand the specification you want to implement (enough to make a rough estimate of how long you think it might take you to implement, anyway) before considering the case closed.

2024-07-15 10:28 gav: I'll put it another way: if you cannot get far enough through the paper to understand how much effort it might take you or your team to implement, perhaps that's not a great sign for things to come? Basing your predictions not on the specification of what you'd have to implement or you/your team's skills, talents and background but rather on the casual opinion of a single self-selected member of the public seems like an extraordinarily ill-considered approach. (edited)

2024-07-15 10:56 gav: Perhaps one point which could be clearer is the purpose of the JAM-prize: the W3F is not handing $60m+ out specifically in order to get lots of redundant code into the ecosystem. The reason for such a large prize fund is to help create a lot of experts on the protocol. Experts who understand, *inside-out*, both the strict protocol _and_ the numerous factors in creating a viable implementation. The prizes denominated in DOT, Fellowship membership and other long-term benefits and support are all there to help incentivise those experts to stick around for the long-haul whether to continue improving their implementations, to help evolve the protocol or to build atop of it. In pursuit of this, we'll take a rather sink-or-swim tactic with teams: beyond corrections and clarifications to the GP, we'll be offering no great help along the way and will expect teams to solve their own problems in their own way. It's not the fastest way of getting from A to B, but this forges a rare asset: the kind of expertise you get from innovators learning through first-principles. Furthermore, it also helps ensure an intellectually diverse ecosystem less constrained by the biases and design-decisions of a single engineering team, leading to a more resilient engineering ecosystem. (edited)

2024-07-15 11:00 gav: They say there are no Dumb Questions. Perhaps. But there are certainly Lazy Questions. If any come my way I'll mark them LQ and move on. Be warned :P (edited)

2024-07-15 11:04 masapr: I'm sorry, I really didn't want to upset you. Yes, it certainly was a lazy question. But I still believe, it's a fair question. These things are hard to estimate and it's not dumb to ask the expert.

2024-07-15 11:04 gav: The point is to make you the expert :)

2024-07-15 11:04 gav: Without that expertise, you may find the prize difficult to collect: it's certainly the intention.

2024-07-15 11:07 prematurata: Hello, I submitted the application form for section C, specifically JavaScript. Is JavaScript vanilla required or can the code be developed in something more robust like TypeScript which is a superset of JS?

2024-07-15 11:07 gav: Major dialects are generally fine. TypeScript certainly. (edited)

2024-07-15 16:02 erkt: I know things like some crypto libs such as the RingVRF are FFI eligible. Are projects that FFI into the PVM eligible as well? Or is every impl expected to implement the PVM?

2024-07-15 16:32 gav: Every project is expected to implement. However, since M1 and M2 do not have any performance requirements, you can get away with implementing a simple interpreter, which shouldn't take too long (Jan managed to implement it in less than 24 hours). (edited)

2024-07-16 09:54 dylanverstraete: Hi everyone, is there a team working on an implementation in Go? I could help out. Otherwise, who wants to start a team?

2024-07-16 10:03 darkartistry: > <@dylanverstraete:matrix.org> Hi everyone, is there a team working on an implementation in Go? I could help out. Otherwise, who wants to start a team? Hi I’d like to work in a team. Have you read the paper ?

2024-07-16 12:35 oliver.tale-yazdi: I create this page so that teams can be peripherally aware of each other's existence: https://jamcha.in/clients It should help to discover under/over-saturated language(sets) to better spread out our efforts. It is an unofficial page though, so its not at all required 😄 (edited)

2024-07-16 12:46 darentuzi: > <@dylanverstraete:matrix.org> Hi everyone, is there a team working on an implementation in Go? I could help out. Otherwise, who wants to start a team? yeah we have a team of 5 working on it right now

2024-07-16 12:48 dylanverstraete: > <@darentuzi:matrix.org> yeah we have a team of 5 working on it right now Alright good to hear, good luck!

2024-07-16 12:54 dakkk: > <@oliver.tale-yazdi:parity.io> I create this page so that teams can be peripherally aware of each other's existence: https://jamcha.in/clients > It should help to discover under/over-saturated language(sets) to better spread out our efforts. It is an unofficial page though, so its not at all required 😄 cool; I submitted mine

2024-07-16 15:00 ainhoa_a: same, submitted too, Jam jammy, thanks for the initiative!

2024-07-16 16:01 eclesiomelo:

2024-07-16 16:03 eclesiomelo: hello everyone! I've started a zig implementation of scale-codec thinking in expand the implementation possibilities, the decoding is not fully implemented tho. Would be nice if fellow zig coders could take a look and provide feedbacks :) here's the repo https://github.com/EclesioMeloJunior/zig-scale-codec

2024-07-16 17:23 sourabhniyogi: Devcon 7 https://tickets.devcon.org/ went on sale (Nov 12-15th) today -- It would be awesome to have a "JAM0" weeklong hackathon (like 9th-17th) at some venue for teams to try to pass Milestone 1 together and (with some luck) have us all trying to get our QUIC networking working with each other, fill in our knowledge gaps, have clear todos for Milestone 2. If this is reasonable, how can people get their travel (and $599 Devcon ticket) reimbursed to such an event? Should this event be OpenGov or W3F funded? What would be some good criteria to get people a $2K voucher to attend?

2024-07-16 18:09 gav: W3F will probably not arrange any official Jam presence at Devcon unless it's clear most serious implementors plan to be there anyway. There might be some co-hosting with Sub0, but it’s TBC. Definitely do want to arrange some sort of seminar conference. Of course you’re free to go to the treasury yourself. (edited)

2024-07-16 19:47 sourabhniyogi: Is this reasonable? https://polkadot.polkassembly.io/post/2397 I am not sure how we could objectively know who are "serious implementors" for travel reimbursement, perhaps there is some "Block Importer" test that could be objectively or subjectively applied? We need some protocol engineers / fellows / W3F, maybe in a 1:10 ratio to implementers for this to fly, I think. (edited)

2024-07-17 07:41 philip.poloczek: Thanks Olivier, this is a great addition! I'm the maintainer of the graypaper.com website. In the initial draft of the website we had an Implementation section like this in mind. We removed it for the launch as it was just empty. Would it be fine for you and everyone, if I add it to the graypaper website based on the teams already collected on https://jamcha.in/clients? (edited)

2024-07-17 08:18 ainhoa_a: > <@philip.poloczek:parity.io> Thanks Olivier, this is a great addition! > > I'm the maintainer of the graypaper.com website. In the initial draft of the website we had an Implementation section like this in mind. We removed it for the launch as it was just empty. > > Would it be fine for you and everyone, if I add it to the graypaper website based on the teams already collected on https://jamcha.in/clients? fine for us let us know if we need to submit a PR or if the info on the site is enough

2024-07-17 10:45 oliver.tale-yazdi: > <@philip.poloczek:parity.io> Thanks Olivier, this is a great addition! > > I'm the maintainer of the graypaper.com website. In the initial draft of the website we had an Implementation section like this in mind. We removed it for the launch as it was just empty. > > Would it be fine for you and everyone, if I add it to the graypaper website based on the teams already collected on https://jamcha.in/clients? yep sounds good, will dm

2024-07-17 13:37 gav: Here's my (unofficial) summary of the prize rules - I'll try to keep it up to date with any questions/answers in here: https://hackmd.io/@polkadot/jamprize

2024-07-17 15:54 philip.poloczek: fyi Oliver and I synced on the already submitted teams/clients for jamcha.in and now share a single source of truth for the data available on both sites: https://jamcha.in/clients https://graypaper.com/clients (edited)

2024-07-17 16:05 prematurata: hello, hopefully not a stupid question. about C.1.5 or Bit Sequence Encoding or (278). variable length sequence. There is no formal definition in the bit sequence encoding so i guess that it's going to be Discriminator Encoding + bitsequenceencoding? (edited)

2024-07-17 17:20 gav: > <@prematurata:matrix.org> hello, hopefully not a stupid question. about C.1.5 or Bit Sequence Encoding or (278). > variable length sequence. There is no formal definition in the bit sequence encoding so i guess that it's going to be Discriminator Encoding + bitsequenceencoding? are you referring to this?...

2024-07-17 17:20 gav: image.png

2024-07-17 17:20 gav: b is a bit sequence - this formula provides the byte sequence which is the encoding.

2024-07-17 17:37 prematurata: Sorry yes, I am referring to that. It says that in case of variable length sequences, then the length is prefixed as in the general case. I am not sure what the "general case" is. So I was speculating that in case of variable length, the "discriminator encoding" defined above would need to be used

2024-07-17 17:42 gav: Yeah, that sentence is a bit superfluous.

2024-07-17 17:43 gav: There are no variable length sequences afair.

2024-07-17 17:43 gav: And where there are I’m careful to make an explicit length prefix.

2024-07-17 17:43 gav: So you can basically ignore it.

2024-07-19 12:45 asmie: Hello. A quick clarification: We can use third-party libraries for cryptographic primitives, codecs, and networking. Does rule #4 (idiomatic code) apply to the external code or our work only?

2024-07-19 14:57 gav: Your work only.

2024-07-20 13:58 sourabhniyogi: @room: JAM implementors! If you would like to have a 4-7 day "JAM0" implementers room @ Devcon 7 (in Bangkok) where you and other serious implementers network your/our implementations together \[while following the official / unofficial Rules\], please comment here: https://polkadot.polkassembly.io/post/2397 and get listed here: https://jamcha.in/clients A simple comment like this: https://polkadot.polkassembly.io/post/2397#Zgpa0MJvU0LsDITa5kHe from 3-4 more serious teams would be sufficient to justify getting an OpenGov-supported venue. (edited)

2024-07-20 18:06 sisco0: I am interested at joining a C team of Jam. How could I do it? :) Thank you! I could observe only Rust implementation at https://jamcha.in/clients (edited)

2024-07-21 07:52 gav: > <@sisco0:matrix.org> I am interested at joining a C team of Jam. How could I do it? :) Thank you! > I could observe only Rust implementation at https://jamcha.in/clients Perhaps you should start one:)

2024-07-22 14:25 sisco0: Thank you gav , I would set a plan for the current C implementation of Jam.

2024-07-23 12:35 sourabhniyogi: @room: JAM implementors, thank you for showing interest! We have a quorum of JAM implementors -- I posted this accordingly: https://polkadot.polkassembly.io/referenda/1024 I am coordinating with yaxun@OneBlock on securing a venue (and food arrangements) [see OpenGov Ref 897], hope we can report some good news soon.

2024-07-23 20:34 rmagon:

2024-07-23 20:36 rmagon: Hello! I'm from the team behind airlyft & we're planning to start a client implementation but wondering what would be the best way forward in terms of the language? My first thoughts were Go, Rust, Typescript but I see we already have some implementations working on those languages. Another option would be C++. In terms of our strengths, these are the languages we feel we can make a difference with. But in terms of the prize pool, does a different language in the same set make a difference? Any suggestions? (edited)

2024-07-23 21:14 rmagon: > <@rmagon:matrix.org> Hello! I'm from the team behind airlyft & we're planning to start a client implementation but wondering what would be the best way forward in terms of the language? My first thoughts were Go, Rust, Typescript but I see we already have some implementations working on those languages. Another option would be C++. In terms of our strengths, these are the languages we feel we can make a difference with. But in terms of the prize pool, does a different language in the same set make a difference? Any suggestions? Okay, I think I got the answer to my question. It's the Language Set which makes a difference and not the language. Right?

2024-07-25 22:55 gatesma: [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]][[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]][]]]]]]]]]][[]][[]][[][]][[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]][][][[][][][][][][][][][][][][][][][][][][][][][][][][][][][][]=][

2024-07-25 22:55 gatesma: [][][][][][][][]

2024-07-25 23:00 gatesma: sorry, my son is a bit wild on the keyboard.

2024-07-25 23:07 xlchen: oh it is not some private key encoded with brainf*ck?

2024-07-26 13:00 tomusdrw: > <@sourabhniyogi:matrix.org> @room: JAM implementors, thank you for showing interest! We have a quorum of JAM implementors -- I posted this accordingly: > https://polkadot.polkassembly.io/referenda/1024 > > I am coordinating with yaxun@OneBlock on securing a venue (and food arrangements) [see OpenGov Ref 897], hope we can report some good news soon. Hey! Is there still a chance to join? 3 typeberry developers would be interested to come along as well.

2024-07-26 14:08 danicuki: Maybe this is a dumb question, but I decided to ask anyway... I noticed that most implementers decided to keep their evolution and code private for now (https://github.com/w3f-webops/graypaper-website/blob/main/src/data/clients.json). Why?

2024-07-26 14:19 rmagon: I had the same question, tbh but did not ask :D I'm guessing it's due to the clean room principles ? (edited)

2024-07-26 17:18 gav: I have previously proposed that the Polkadot JAM chain should not be launched until there are at least 3 implementations which perform adequately for mainnet (ie pass Milestone 5). This doesn’t affect the prize in any way.

2024-07-26 21:50 sourabhniyogi: For Rule 19 - "Prizes are paid no earlier than the ratification by the Polkadot Fellowship of version 1.0 of the JAM protocol. Payment of the prize by the Web3.0 Foundation is conditional upon the successful completion of all KYC/AML processes." -- what are the preconditions that need be satisfied for you (or the Fellowship) to decide to ratify version 1.0 of the JAM protocol? Could version 1.0 of the JAM protocol be ratified with 0 teams passing Milestone 1 -- where ratification is _just_ about the GP? Would a single implementation team passing Milestone 1 or 2 or 3 be sufficient to trigger 1.0 ratification? If so, which milestone? Would a N implementation teams passing Milestone 1 or 2 or 3 be sufficient to trigger 1.0 ratification? If so, for what N and what milestone?

2024-07-27 14:09 gav: > <@sourabhniyogi:matrix.org> For Rule 19 - "Prizes are paid no earlier than the ratification by the Polkadot Fellowship of version 1.0 of the JAM protocol. Payment of the prize by the Web3.0 Foundation is conditional upon the successful completion of all KYC/AML processes." -- what are the preconditions that need be satisfied for you (or the Fellowship) to decide to ratify version 1.0 of the JAM protocol? > > Could version 1.0 of the JAM protocol be ratified with 0 teams passing Milestone 1 -- where ratification is _just_ about the GP? > Would a single implementation team passing Milestone 1 or 2 or 3 be sufficient to trigger 1.0 ratification? If so, which milestone? > Would a N implementation teams passing Milestone 1 or 2 or 3 be sufficient to trigger 1.0 ratification? If so, for what N and what milestone? > The preconditions are not stated and it is the Fellowship’s decision to ratify the protocol specification when it believes it ready.

2024-07-27 14:09 gav: Realistically, it will need to be stable, useful, capable of hosting a service compatible with the relay chain and secure. The latter point will imply an external audit.

2024-07-27 14:11 gav: However that’s just a rough guide. The Fellowship’s decision is final (notwithstanding Root-level governance, of course).

2024-07-27 14:58 sourabhniyogi: Is the Fellowship ratification of the protocol specification just about ratifying written specifications of (a) GP alone (b) GP + JAMSNP (c) GP + JAMSNP + documentation of how the service compatible with the relay chain will work or (d) something else ? Is the external audit about GP + \_\_\_ written specifications (one of (a)-(d)) or about an implementation? A serious concern is that serious implementers may (very likely) pass Milestone 1 + 2 + .. in early 2025 handily, but, because of unknown ratification timing they aren't winning prizes for their Milestone 1 + 2 + .. until an external audit that would only be completed in late 2025 _or later_. If this is a valid concern, then we may have teams only working on this as a hobby to part-time (\< 20%), and slowing down bringing JAM into the world. Obviously, JAM teams would like prizes paid out when Milestone 1 is completed, when Milestone 2 is completed, ... -- and not be faced with some unstated preconditions of Rule 19. Is there any route to get teams working on JAM implementations full-time, fully incentivized to bring JAM into the world? (edited)

2024-07-27 15:12 gav: The JAM Prize a prize, not a service agreement. (edited)

2024-07-27 15:12 gav: Approaching this as a service agreement is going to result in unrealistic expectations.

2024-07-27 15:14 gav: Fellowship ratification of the JAM protocol would be on the JAM protocol definition which is the Gray Paper.

2024-07-27 15:17 gav: The GP will include JAMNP.

2024-07-27 15:17 gav: This is not a priority at present since it is not required for M1. Furthermore it is the only part of the protocol which is heavily reliant on practical concerns which cannot easily be modelled. It will likely be the very last protocol element to be finalized. (edited)

2024-07-27 15:18 gav: The JAM protocol is independent of any implementation.

2024-07-27 15:19 gav: Most of these answers are plainly obvious. Please note that further questions with obvious answers may elicit a more blunt response. (edited)

2024-07-27 15:56 sourabhniyogi: Thank you for clarifying, everything is now more obvious =)

2024-07-27 16:39 tebdev: > <@philip.poloczek:parity.io> Thanks Olivier, this is a great addition! > > I'm the maintainer of the graypaper.com website. In the initial draft of the website we had an Implementation section like this in mind. We removed it for the launch as it was just empty. > > Would it be fine for you and everyone, if I add it to the graypaper website based on the teams already collected on https://jamcha.in/clients? Client diversity got my attention. I used this at NASA, etc. impressed it being considered for JAM. Bravo

2024-07-28 16:24 gav: Furthermore, canonical M1 test vectors will not be available before 1.0 is ratified.

2024-07-28 16:25 gav: Fellowship is the basis of technical expertise in Polkadot and should generally be relied upon for judgements of a protocol/technical nature.

2024-07-28 16:27 gav: You quoted me from this room and followed it with: “In particular, SCALE-encoding details are likely to change and as such, teams should design to have networking protocol versioning within it for efficient reasoning”.

2024-07-28 16:28 gav: I did not say this nor did I say anything to this meaning.

2024-07-28 16:28 gav: It is not my expectation that the underlying method for serialization/encoding will change and I would urge to refrain from writing such misleading phrases. (edited)

2024-07-28 16:29 sourabhniyogi: Removed, is there anything else misleading?

2024-07-28 16:29 gav: Perhaps you should read it. I am not your fact-checker.

2024-07-28 16:49 sourabhniyogi: On networking, https://hackmd.io/@polkadot/jamsnp is all we have right now, its very new and fresh since you only posted it at the start of this month =). I cannot guess the difference betweeen JAMSNP, JAMNP-S, or JAMNP -- - is there a difference? Its definitely not obvious with a lot of TBDs. If you say the networking will be the one of the last things to be finalized, we understand. On canonical M1 test vectors, is this a different set than what you referred to on June 3: "Block execution vectors should be ~6 weeks out. Availability system test vectors probably about the same timeline." The existence of these test vectors, even in draft PR form, are highly valuable accelerants for M1/M2 well well in advance of 1.0 ratification. But given how well described the GP is, implementers can fill in many details of block import vectors + JAM network details, some/most of which are easy to infer from the GP alone. The more complete the test vectors are, and closer to the "prize" requirements, the better, but I understand pestering won't help. We imagine honest test vectors can come from the implementers themselves as well, however not canonical. Some of the leading teams have fellowship talent within their ranks, and we understand its a side goal to develop fellowship talent through increased JAM protocol expertise. Earning your rank could be done through this kind of contribution. (edited)

2024-07-28 17:09 emielvanderhoek: If you ask me more/alternate funding in addition to the prize leads to poor incentives and attracts the wrong people. Also cringe: “serious implementers”. (edited)

2024-07-28 17:38 sourabhniyogi: The notion of "serious implementers" comes from "W3F will probably not arrange any official Jam presence at Devcon unless it's clear most serious implementors plan to be there anyway." Perhaps that will make some people cringe, but the notion of what constitutes "serious" is difficult to discern from observing https://jamcha.in/ right now. I gather it is "you will stick around after completing M5 and claiming the prize, and become a valued fellowship member" like a missionary instead of "We will leave after claiming X milestone bounty" like a mercenary. Can anyone spot the mercenaries in https://jamcha.in/ now without ever having interacted with them? The mercenaries within the JAM implementer set are those that are just like "what do we have to do to pass Milestone X and get Milestone X prize money (or bounty)". The missionaries want to build the whole thing, are curious as to how and why it works, and will want to improve it/attack it/... Interviewers can often detect the difference. Unfortunately, its hard for bounty curators, JAM Prize judges or any human to read peoples minds objectively the way you can objectively check whether some implementation does or does not pass a test vector. Every incentive structure will have some type 1 and type 2 "errors", if you can call them "errors". After all, some "missionaries" will have kids to feed, mortgages to pay off, parents to take care of -- and some "mercenaries" can become "missionaries" after falling in love with the protocol and proving to themselves that they are good at implementing and maintaining it. Its easy to be black-and-white about it and pretend that this kind of real life doesn't exist. (edited)

2024-07-28 18:31 emielvanderhoek: Well, call me a purist. “Serious implementers” is a silly notion period. There’s only “implementers”. It is: conformance, performance & security and in that particular order. Some teams will finish first and some teams will finish later. Any additional incentive structure distorts the objectives of Jam Implementers Prize and unlevels the playing field. I’d suggest a new moderation rule for this channel: to disallow any talk about alternate/additional funding for JAM implementation teams (besides the Prize) in this channel. If we don’t I fear this channel may become a toxic as some of the other channels. You can do what you want in the regular governance channels of course. My 2cts. (edited)

2024-07-28 21:43 emielvanderhoek: “More money will make JAM arrive sooner.”: This to me seems like a big MISconception.

2024-07-29 04:05 xlchen: I am confused about trap vs halt vs panic in PVM. In GP, the trap instruction will exit with the black square, so does the jump to 2^32-2^16 address. To my understanding that is exit 0. But in the pvm testvector, trap is panic and the test of the trap instruction will result trap exit status but the inst_ret_halt test results halt. There is some inconsistency. https://github.com/w3f/jamtestvectors/blob/a2b18702aac7d15b9f51cd1ffcf0be95f987b2f7/pvm/schema.asn#L53-L55

2024-07-29 06:49 xlchen: Screenshot 2024-07-29 at 6.49.40 PM.png

2024-07-29 06:50 xlchen: this means jump_ind could be used to exit the program, but how about the one using branch? e.g. jump

2024-07-29 06:51 xlchen: Screenshot 2024-07-29 at 6.50.56 PM.png

2024-07-29 06:51 xlchen: does it become panic?

2024-07-29 13:24 mkalohood: > <@emielvanderhoek:matrix.org> Well, call me a purist. > “Serious implementers” is a silly notion period. There’s only “implementers”. It is: conformance, performance & security and in that particular order. Some teams will finish first and some teams will finish later. > Any additional incentive structure distorts the objectives of Jam Implementers Prize and unlevels the playing field. > I’d suggest a new moderation rule for this channel: to disallow any talk about alternate/additional funding for JAM implementation teams (besides the Prize) in this channel. > If we don’t I fear this channel may become a toxic as some of the other channels. > You can do what you want in the regular governance channels of course. > My 2cts. It is strange now to consider what is called serious implementers before enough participants. Any good ecosystem is profit-driven, and any project it wants to participate in will consider the input-output ratio, rather than competing in an uncertain competition.

2024-07-29 13:52 tebdev: > <@mkalohood:matrix.org> It is strange now to consider what is called serious implementers before enough participants. Any good ecosystem is profit-driven, and any project it wants to participate in will consider the input-output ratio, rather than competing in an uncertain competition. Agreed, "serious" subjective. if anyone contributes to moving the needle; it is a positive contribution and should not be discounted. Incentives is a NASH to accelerate work, but if you "serious" about blockchain being a movement then "one" would encourage participation and teamwork for democratization as the ultimate payoff. Strength in Numbers.

2024-07-29 17:17 tebdev: > <@xlchen:matrix.org> I created some diagrams, will love a review https://hackmd.io/0gSmXyElT2iKawymS5jJKw?view Very nice. I started late, but this is an equalizer... Bravo

2024-07-30 10:29 gav: As of PR #52, trap will panic. (edited)

2024-07-30 10:33 gav: > <@gav:polkadot.io> Looks pretty accurate. Actually, there's one slight (probable) anomaly...

2024-07-30 10:33 gav: > Parachain collator submit work to packager Probably the collator would be the packager.

2024-07-30 10:34 gav: Theoretically they might not be, but if you make them different you add a further degree of latency into the pipeline which could delay finality.

2024-07-30 10:46 xlchen: but for more advanced use cases such as synchronous xcmp call, do we not need something like MEV bundler to search for the best way to allocate the work?

2024-07-30 10:47 xlchen: to build multiple parachain blocks in a single work package

2024-07-30 10:48 gav: Yeah, but then each collator would need to be aware that they’re being co-bundled.

2024-07-30 10:49 gav: So we’d need some sort of multi-parachain collator node.

2024-07-30 10:49 gav: Probably a pluggable thing or a standard synchronous-composable parachain collation API.

2024-07-30 10:53 xlchen: I see. Will adjust the diagram when I get the chance.

2024-07-30 10:57 xlchen: we will need to figure out a new collator selection algorithm to maximize number of processed xcmp messages. another thing to think about in future

2024-07-30 11:03 bkchr: > <@xlchen:matrix.org> but for more advanced use cases such as synchronous xcmp call, do we not need something like MEV bundler to search for the best way to allocate the work? But then these parachains will also need to share one core. Probably if you have some profitable MEV that you want to execute, it may be useful. But then the question arises on who pays for the core and do the parachain agree to be bundled. Clearly will require some substantial changes to the parachains to support this I think

2024-07-30 11:06 xlchen: yeah I will imagine something similar flashbots that allow MEV searcher to build bundles from tx pool (across all parachains) and share profits with collator

2024-07-30 11:07 bkchr: The general logic of finding out what to bundle together will also be required for coreplay

2024-07-30 11:07 xlchen: but instead of a single chain MEV, it will be a crosschain and presumably there will be fees for XCMP and that’s another source of income

2024-07-30 11:07 bkchr: But there the idea was that there exists one leader that lays out the plan for all the tasks for the work package producers

2024-07-30 11:12 xlchen: the leader can always delegate the work out just like mining pools. it is unlikely that we can just figure out a optimal algorithm so it is something the implementors can compete

2024-07-30 11:15 bkchr: Yes

2024-07-30 16:11 sourabhniyogi: JAM implementers if you haven't gotten tickets to Devcon yet (and want to go to this: https://polkadot.polkassembly.io/referenda/1024), they went on sale a few minutes back: https://tickets.devcon.org/ They tend to get sold out quickly. There will be another wave but it might be the last one so I ask that you act quickly.

2024-07-30 16:28 sourabhniyogi: We are getting quotes from venues still, but would like a headcount of who is committed to attend -- if you can fill in your team size and # of people who are committed to attend here: https://docs.google.com/spreadsheets/d/1_Ar0CWH8cDq_mAoVkqZ20fXjfNQQ9ziv1jsVJBAfd1c/edit?gid=0#gid=0 That would help us decide whether we get a 30 vs 50 vs 70 person venue. The good news is that the venue costs are lower than expected so we may have bigger travel budgets; the bad news is that we are on waiting lists for the best places and I can't tell you what the venue is quite yet.

2024-07-30 16:37 sourabhniyogi: As some people have had some difficulty adding a comment on Polkassembly [due to identity not being set] feel free to add your name to the sheet. I will create a new room with the key team contacts so we don't get off JAM topics. Thank you!

2024-07-30 20:42 shawntabrizi: I want to present an idea having only really high level understanding of JAM, so I apologize in advance if I am just talking air. My comments basically encapsulates the issue opened on Polkadot here: https://github.com/paritytech/polkadot-sdk/issues/811 As I understand with how services are being designed in JAM, their only ability to read and write data is to the DA layer. This is of course the preferred and most scalable method for services to place data. Services that want to store long term data are asked to have their own data shard. However, when thinking about the kinds of services which can be built on JAM, and affording those services the most flexibility, I can imagine that services will want to be able to write to a long term storage without having to maintain their own collators or data shards. This is basically what existing smart contract platforms afford to contracts. My high level suggestion is that JAM should also include an API where services can write data to the main JAM database, probably under the general opaque key/value store of preimages. Then, executing the state transition function of services should be able to read from a combination of the DA and the stored preimages in the JAM database. Writing here could be super expensive, such that we encourage good behavior to NOT use it frivolously, but to not have it at all seems like a missed scenario. I think if JAM is going to be the most general web3 computer, it does make sense to have such an API, and can help provide a feature which can onboard more services to use JAM. (edited)

2024-07-31 05:37 ltfschoen: > <@shawntabrizi:matrix.org> I want to present an idea having only really high level understanding of JAM, so I apologize in advance if I am just talking air. > > My comments basically encapsulates the issue opened on Polkadot here: https://github.com/paritytech/polkadot-sdk/issues/811 > > As I understand with how services are being designed in JAM, their only ability to read and write data is to the DA layer. > > This is of course the preferred and most scalable method for services to place data. Services that want to store long term data are asked to have their own data shard. > > However, when thinking about the kinds of services which can be built on JAM, and affording those services the most flexibility, I can imagine that services will want to be able to write to a long term storage without having to maintain their own collators or data shards. This is basically what existing smart contract platforms afford to contracts. > > My high level suggestion is that JAM should also include an API where services can write data to the main JAM database, probably under the general opaque key/value store of preimages. > > Then, executing the state transition function of services should be able to read from a combination of the DA and the stored preimages in the JAM database. > > Writing here could be super expensive, such that we encourage good behavior to NOT use it frivolously, but to not have it at all seems like a missed scenario. > > I think if JAM is going to be the most general web3 computer, it does make sense to have such an API, and can help provide a feature which can onboard more services to use JAM. > As I understand with how services are being designed in JAM, their only ability to read and write data is to the DA layer. in contrast to Polkadot where "parachains's code never read from the DA layer" source: https://blog.kianenigma.nl/posts/tech/demystifying-jam/

2024-07-31 08:37 gav: > <@shawntabrizi:matrix.org> I want to present an idea having only really high level understanding of JAM, so I apologize in advance if I am just talking air. > > My comments basically encapsulates the issue opened on Polkadot here: https://github.com/paritytech/polkadot-sdk/issues/811 > > As I understand with how services are being designed in JAM, their only ability to read and write data is to the DA layer. > > This is of course the preferred and most scalable method for services to place data. Services that want to store long term data are asked to have their own data shard. > > However, when thinking about the kinds of services which can be built on JAM, and affording those services the most flexibility, I can imagine that services will want to be able to write to a long term storage without having to maintain their own collators or data shards. This is basically what existing smart contract platforms afford to contracts. > > My high level suggestion is that JAM should also include an API where services can write data to the main JAM database, probably under the general opaque key/value store of preimages. > > Then, executing the state transition function of services should be able to read from a combination of the DA and the stored preimages in the JAM database. > > Writing here could be super expensive, such that we encourage good behavior to NOT use it frivolously, but to not have it at all seems like a missed scenario. > > I think if JAM is going to be the most general web3 computer, it does make sense to have such an API, and can help provide a feature which can onboard more services to use JAM. The DA system is not the only storage available. There is also the service state.

2024-07-31 10:08 tebdev:

2024-07-31 13:00 tebdev: Please excuse, trying to set a tempo for my team. is the goal to have M1 - Dec, M2 - ???, etc.? If documented somewhere, please help me with a URL.

2024-07-31 14:10 gav: See the chat around here. (edited)

2024-07-31 14:10 gav: > <@tebdev:matrix.org> Please excuse, trying to set a tempo for my team. is the goal to have M1 - Dec, M2 - ???, etc.? If documented somewhere, please help me with a URL. See ^^^

2024-07-31 15:32 sourabhniyogi: For those of you who are trying to write their first refine / accumulate code (or connect PVM + Patricia Merkle Tries + DA to your first attempt at host functions), this assembly compiler is going to be insanely useful:

# cat tools/spectool/spec/src/inst_branch_greater_or_equal_signed_ok.txt 
pub @main:
    a0 = 10
    a1 = -10
    jump @target if a0 >=s a1
    trap
@target:
    a0 = 0xdeadbeef

# cargo run -p polkatool -- assemble tools/spectool/spec/src/inst_branch_greater_or_equal_signed_ok.txt -o output.polkavm
# cargo run -p polkatool disassemble --show-raw-bytes output.polkavm
// RO data = 0/0 bytes
// RW data = 0/0 bytes
// Stack size = 0 bytes

// Instructions = 5
// Code size = 16 bytes

      :                          @0 [export #0: 'main']
     0: 04 07 0a                 a0 = 0xa
     3: 04 08 f6                 a1 = 0xfffffff6
     6: 2b 87 04                 jump 10 if a0 >=s a1
      :                          @1
     9: 00                       trap
      :                          @2 [export #1: 'target']
    10: 04 07 ef be ad de        a0 = 0xdeadbeef

This is from beloved @koute https://github.com/w3f/jamtestvectors/pull/3#issuecomment-2257688558 -- having this assembler spares everyone from hand-assembling PVM byte code like its 1964, but at least for now we have to figure out the closest thing to read/write/solicit/lookup/historical\_lookup/import/export and do a bit of hand assembling. But it won't be _too bad_. (edited)

2024-07-31 18:01 tebdev: > <@gav:polkadot.io> See ^^^ Perfecto. Expert is greater payoff than money. Money is easy. LQ --> LA (answer) ;)

2024-07-31 22:19 danicuki: I was thinking about organizing a JAM meetup in Lisbon. Would you come? Register here: https://lu.ma/fqg0ktn0

2024-08-01 13:22 ltfschoen:

2024-08-01 13:38 ltfschoen: > <@danicuki:matrix.org> I was thinking about organizing a JAM meetup in Lisbon. Would you come? Register here: https://lu.ma/fqg0ktn0 i think you need to have rules for attendees like "Everyone will be required to follow the official W3F rules and unofficial "Rules" (from Gav)" as shown in https://polkadot.polkassembly.io/referenda/1024

2024-08-01 13:44 gav: There's really nothing problematic about meeting; my suggestions are there to help ensure that nobody is accidentally disenfranchised. Ultimately, if you can prove clear expertise over the code you're writing, and explain all design and interpretation decisions, then there really shouldn't be a problem. Just be sure to document and declare any non-trivial "coordinated learning"! (edited)

2024-08-01 16:43 s_saxemberg: Hello Jam implementers, we just wanted to invite you to let us know (the community and tokenholders) more about the identities of the Jam implementers that would be the beneficiaries of referendum 1024 https://polkadot.subsquare.io/referenda/1024 (trip to Thailand Devcon7) allegedly this will benefit +15 teams and many more people we would like to meet. We are reaching to you guys through this channel because this is the linked chat that was provided on the referendum. Thank you and sorry for the unrelated interruption.

2024-08-01 18:33 celadari: Hi davxy , I checked the chat and read your discussion with Sourabh regarding signature verification. I also checked the two repositories ark-ec-vrfs and bandersnatch-vrfs-spec I wanted to confirm my understanding: Is pub type RingCommitment<S> = ring_proof::FixedColumnsCommitted<BaseField<S>, PcsCommitment<S>>; the ring root commitment ($\gamma_z$) mentioned in the Gray Paper? If yes, when we need a verifier for the ring commitment, we need PcsParams. What should I use? From the code, it seems we take these params from the ring context, and there appears to be some randomness when instantiating ring context. Is this normal? Should I just initialize the ring context with RingContext::from_seed(RING_DOMAIN_SIZE, [0; 32]) in the code and not worry about the rest ? Thanks in advance !

2024-08-02 05:56 xlchen: Any suggestions on in what order to implement the protocols? Considerations: stability of the spec, availability and stability of test vectors, coupling with other protocols (e.g. can't do B unless A is implemented and size/complexity. e.g. First thing should be the codec, and then Safrole (which depends on the codec) is a good starting point as it is relatively simple, presumably no plan to update it in GP, and with test vectors merged (i.e. stable). and then?

2024-08-02 05:57 xlchen: I see open PR on the testvectors so that means they are available but because not yet merged I have no idea on how stable they are.

2024-08-02 05:59 xlchen: I am a bit hesitated on working on the protocols without test vectors because I could do it completely wrong and won't be able know until related test vectors are released

2024-08-02 07:55 asmie: Hi! qq about in-memory db - can we use anything external (eg.: RocksDB, ParityDB) or it should be something implemented from scratch?

2024-08-02 15:07 gav: > <@xlchen:matrix.org> I am a bit hesitated on working on the protocols without test vectors because I could do it completely wrong and won't be able know until related test vectors are released The protocol definition is 0.3.2. There are going to be changes. Nothing is final.

2024-08-03 15:13 ltfschoen: does anyone know where i can find out more information about the **zkSNARK Service** that Gav mentioned at 51mins in this old discussion "Gavin Wood on JAM: The next disruptor in Web3" https://x.com/paritytech/status/1788847203622043974 (edited)

2024-08-05 15:19 tebdev: > <@asmie:matrix.org> Hi! qq about in-memory db - can we use anything external (eg.: RocksDB, ParityDB) or it should be something implemented from scratch? This a good question. I also need advice.

2024-08-06 09:35 dudley_kilt: Dudley from the KILT Protocol team. We are currently putting a team of Parachain members to build a JAM Client in Rust. We are looking to comprise multiple ecosystem partners of a team around 4-6 people either part time or full time that would implement the graypaper. We would be beginning in the following weeks. We were wondering the state of play considering the stiff competition in the B category. Our goal is to build knowledge from the existing ecosystem teams and share that expertise of the JAM Clients to build a stronger ecosystem when the JAM comes around consider it “Spreading the butter over the toast then the JAM .” We are looking for team members in the ecosystem Rust from Parachain teams to include. KILT would be leading the team.

2024-08-06 10:37 darentuzi: > <@asmie:matrix.org> Hi! qq about in-memory db - can we use anything external (eg.: RocksDB, ParityDB) or it should be something implemented from scratch? Can anyone help clarify this, my guess is that an existing DB should be fine?

2024-08-06 10:47 subotic: > <@darentuzi:matrix.org> Can anyone help clarify this, my guess is that an existing DB should be fine? This is one of previous Gav's responses: "Third-party libraries (FFI) for cryptographic primitives (erasure-coding, Bandersnatch, Ed25519), codecs (e.g. SCALE) and networking (e.g. QUIC) are acceptable. Everything else should be properly implemented." So from that I understand that DB falls under "everything else".

2024-08-06 17:31 gav: > <@subotic:matrix.org> This is one of previous Gav's responses: "Third-party libraries (FFI) for cryptographic primitives (erasure-coding, Bandersnatch, Ed25519), codecs (e.g. SCALE) and networking (e.g. QUIC) are acceptable. Everything else should be properly implemented." > > So from that I understand that DB falls under "everything else". Sorry missed this. It’s fine to use a third-party DB.

2024-08-06 17:32 gav: Same goes for anything similarly generic.

2024-08-06 20:43 erkt: Point of Clarification: The latest version of the GP removes any mention of the old SCALE codec, however, in some convos/threads in the chat, SCALE is still being mentioned. I know the variable integer encoding scheme in GP is already different from the compact integer encoding in SCALE. So is it safe to say that SCALE will not be used in JAM?

2024-08-06 20:46 erkt:

2024-08-06 20:55 erkt: Also, on the note of Integer encoding, I'm having a bit of confusion with the 29-bit serialization rules (Eqn 274). Did the writer mean to say the 3rd condition is: "otherwise if x <2^29" similarly to Eqn 275? Alternatively should l be in N_2 instead of N_3?

2024-08-07 09:38 gav: > <@erkt:matrix.org> Point of Clarification: The latest version of the GP removes any mention of the old SCALE codec, however, in some convos/threads in the chat, SCALE is still being mentioned. I know the variable integer encoding scheme in GP is already different from the compact integer encoding in SCALE. So is it safe to say that SCALE will not be used in JAM? Yes.

2024-08-08 19:21 celadari: Is the size in bytes of gamma_z always the same ? If not, is there a limit on the number of bytes of gamma\_z Or a way to know it in advance before computing it ? (I ask this because we use a ffi to execute the rust code ark-bandersnatch and to get the result back from our language we provide a pointer with free memory and the length of this memory) (edited)

2024-08-08 19:31 celadari: So far I've only seen gamma_z with size of 144 bytes in the test vectors but I wanted to confirm the size in bytes of gamma_z instead of assuming (edited)

2024-08-08 19:32 prematurata: it's 144

2024-08-08 19:34 prematurata: you can confirm it also by checking appendix I which states Y_r is a subset of Y_144 and if u check (49) then gamma_z is defined to be in Y_r

2024-08-09 11:41 vinsystems: In the test enact-epoch-change-with-no-tickets-4.json of Safrole, the value of the parameters in post_state indicates that there has been an epoch change (e' > e). However, if I calculate the value of e and e', they are the same... e = tau / E; e = 10 / 600 = 0 e' = tau' / E; e' = 15 / 600 = 0 Therefore, e' = e. What I missing here? https://github.com/w3f/jamtestvectors/blob/master/safrole/tiny/enact-epoch-change-with-no-tickets-4.json

2024-08-09 11:44 xlchen: https://github.com/w3f/jamtestvectors/blob/master/safrole/tiny.asn#L4

2024-08-09 16:25 celadari: Hello, I have a doubt regarding **ordering of extrinsic** in file tiny/publish-tickets-with-mark-2.json. We have the following extrinsic:

{
        "attempt": 1,
        "signature": "0x8e.......cf7"
      },
      {
        "attempt": 0,
        "signature": "0x38.....92"
      },
      {
        "attempt": 0,
        "signature": "0xa7....38"
      },
      {
        "attempt": 1,
        "signature": "0x52....f0"
      },
      {
        "attempt": 1,
        "signature": "0x16......53"
      }

Verifying signatures and thus getting VRF output I get in the same order: 0x3a....ad 0x5d....98 0x8b....d9 0xae....b4 0xb1.....75 but then they are not in any order (ascending or descending) if taken as LITTLE-ENDIAN. (They are in ascending order if taken as BIG-ENDIAN though...) **Is there a problem with the order of extrinsic in this file or there is something I am missing ?** (edited)

2024-08-12 11:00 dakkk:

2024-08-12 11:28 dakkk: Regarding the definition of Y in appendix G, the two functions "decode" and "hashed_output" are referenced; where are defined? I can't find a valid "name match" in the linked papers

2024-08-14 17:48 davxy: ⚠️ First proposal for Disputes STF test vectors (Section 10) : https://github.com/w3f/jamtestvectors/pull/9 Be aware, it may not be final, and some adjustments might be made. However, you can start using it.

2024-08-15 07:23 bkchr: > <@sourabhniyogi:matrix.org> JAM implementers if you haven't gotten tickets to Devcon yet (and want to go to this: https://polkadot.polkassembly.io/referenda/1024), they went on sale a few minutes back: > https://tickets.devcon.org/ > They tend to get sold out quickly. There will be another wave but it might be the last one so I ask that you act quickly. With the recent proposal of having a sub0 next to devcon, this event should be scrapped and included in sub0. The sub0 proposal is any way about having a hackathon style event thst wants to attract different kind of peoples. First, this will reduce costs for the ecosystem and secondly it will also have a much bigger impact if the ecosystem can participate.

2024-08-15 07:25 bkchr: This was also directly proposed by rtti-5220 as comment to the proposal: https://polkadot.polkassembly.io/referenda/1024#GJjjWhHPJSlZt2NObHFO

2024-08-15 07:25 gav: > <@bkchr:parity.io> With the recent proposal of having a sub0 next to devcon, this event should be scrapped and included in sub0. The sub0 proposal is any way about having a hackathon style event thst wants to attract different kind of peoples. First, this will reduce costs for the ecosystem and secondly it will also have a much bigger impact if the ecosystem can participate. When you say “next to”, do you mean in time or space?:)

2024-08-15 07:26 bkchr: Time and space.

2024-08-15 07:27 gav: Ahh right - actually during devcon and in Bangkok. (edited)

2024-08-15 07:30 bkchr: https://forum.polkadot.network/t/sub0-reset-devcon-bangkok-back-to-the-roots/9626

2024-08-15 07:31 bkchr: 9-11 November. Devcon starts at the 12th

2024-08-15 07:31 rtti-5220: > <@gav:polkadot.io> Ahh right - actually during devcon and in Bangkok. which is the case for this proposal, but i think something in parallel to sub0, same venue, 1 day completely focused on JAM would make more sense for JAM impl devs who can attend both events easily (sub0 and Jam0), and it would reduce costs (instead of organising 2 separate events in different parts of the city, we get two events in one) - we are also experimenting with this first JAM event so something in a more controlled enviroment like Sub0 and getting support from the Sub0 organisers might make more sense.

2024-08-15 07:33 gav: Perhaps we could run a parallel event in Europe at the same time.

2024-08-15 07:40 emielvanderhoek: Although it may not be official I believe lots of JAM teams are making preparations for attending JAM0 as a side event for Devcon in Bangkok.

2024-08-15 07:42 emielvanderhoek: 15 teams already last I read.

2024-08-15 07:43 gav: Out of 23, but if the other 8 happen to be within easy travelling distance of Lisbon then it could be nice to host something small at the Parity office there.

2024-08-15 07:43 gav: Cost would be minimal compared to the 100k being asked for Jam0 Bangkok.

2024-08-15 07:44 emielvanderhoek: Sure it would be nice to have a shared agenda/objective at least.

2024-08-15 07:44 gav: And personally I probably can’t make Bangkok in November. (edited)

2024-08-15 07:53 emielvanderhoek: The assumption I believe is that half November we’d get as far as running a M1 ‘tiny’ importer network between clients. Timing at least makes this reasonable perhaps given it is three month ahead of us.

2024-08-15 08:00 bkchr: Also JAM could be helpful to "lure in" more people to sub0. Getting actually people from ETH interested on JAM and through this learning some stuff about the current stack could be quite helpful for the entire ecosystem

2024-08-15 12:09 sourabhniyogi: Guys, thank you for entertaining the JAM0 event. It was about travel budgets for like 38 people, which are not covered under the sub0 proposal, which has dates which are too early for most people to sustain a 9-10 day trip. I hope JAM implementers didn't buy Devcon 7 tickets and nonrefundable airfare. (edited)

2024-08-15 12:15 bkchr: The sub0 proposal is not up yet and they would be happy to extend the proposal by including the proposed travel budgets

2024-08-15 12:15 sourabhniyogi: Let us consider 1024 JAM0 scrapped. If you bought tickets/airfare/hotel in expectation that it would be covered by 1024, please get them refunded.

2024-08-15 12:17 bkchr: I don't get why you want to scrape it?

2024-08-15 12:20 sourabhniyogi: https://polkadot.polkassembly.io/referenda/1024 won't pass at this stage?

2024-08-15 12:21 bkchr: Yeah, but just let it integrate into sub0. Why not?

2024-08-15 12:24 sourabhniyogi: Some of us literally can't make 9th/10th ( In particular, me ). Here is the list of teams for Roman to work with: https://docs.google.com/spreadsheets/d/1_Ar0CWH8cDq_mAoVkqZ20fXjfNQQ9ziv1jsVJBAfd1c/edit?gid=0#gid=0

2024-08-15 12:27 bkchr: You can still join on the 11th as you had planned before

2024-08-15 12:27 bkchr: I mean this is not really about one single individual, but instead of getting some benefit for the entire ecosystem

2024-08-15 12:32 sourabhniyogi: Is Roman / WebZero in this room? Or, Basti.await can you lead the JAM portion of the sub0 event and get the travel budget added and approved for people? [Or delegate to someone who wants to lead it]

2024-08-15 12:37 sourabhniyogi: Hi rektormoon thanks for joining =)

2024-08-15 12:37 rektormoon: Hey everyone, glad to be here

2024-08-15 12:39 sourabhniyogi: We would like to fuse the JAM0 event concept into sub0 which I understand is on the 9th-11th -- Basti.await believes it is reasonable to get some JAM implementer travel budget added in.

2024-08-15 12:40 sourabhniyogi: https://docs.google.com/spreadsheets/d/1_Ar0CWH8cDq_mAoVkqZ20fXjfNQQ9ziv1jsVJBAfd1c/edit?gid=0#gid=0 has the list of interested people from 1024.

2024-08-15 12:40 rektormoon: I think it makes sense to add the program, which ofc would change the nature of your original plan, as we are more interested in showcasing the concepts and progress to ppl not involved with building JAM (yet)

2024-08-15 12:41 rektormoon: Budget wise we don’t really comp travel for speakers and such, but that could also be dealt with by a treasury tip for example. We just don’t really have capacity to be a “travel agency” for idk 30-40 ppl

2024-08-15 12:42 bkchr: The point is more about including the travel budget in the sub0 proposal

2024-08-15 12:43 rektormoon: > <@bkchr:parity.io> The point is more about including the travel budget in the sub0 proposal Ofc, but we would be responsible for that budget

2024-08-15 12:44 bkchr: Okay, I think we can find a way for getting and handling the budget

2024-08-15 12:45 rektormoon: As long as we find good ways to transparently report…

2024-08-15 12:46 rektormoon: At web3 summit we are paying partial travel comp for hackers for the first time, so it’s a new thing for us

2024-08-15 12:46 bkchr: I had misunderstood you, I had thought you had said earlier today that you would be okay to include this in the sub0 proposal. I will figure something out

2024-08-15 12:47 rektormoon: Program wise absolutely!

2024-08-15 12:47 rektormoon: When it comes to payments and budgets, I just want to understand how you see it working and who would be responsible for what :-)

2024-08-15 12:49 sourabhniyogi: Did you lock down an exact venue + dates yet? For pre 11th, you own everything. For 12-17th dates, I can be an assistant to Basti.await .

2024-08-15 12:51 bkchr: I think the general idea would be to move everything earlier

2024-08-15 12:51 gav: > <@gav:polkadot.io> And personally I probably can’t make Bangkok in November. Can't promise anything yet but I might be able to make 9th-11th.

2024-08-15 12:52 sourabhniyogi: To state the obvious, a sub0 without you is barely a sub0 =)

2024-08-15 12:54 rektormoon: That would be sub-par

2024-08-15 12:55 sourabhniyogi: Basti.await: I have a Valia room for the 11th - 16th. https://www.valiahotelbangkok.com/meetings-venues/ This can be a room for JAM Implementers to work after sub0 (assuming 9th-11th) Can you approve this as being addable to the budget? This would be \< $20K (edited)

2024-08-15 13:04 bkchr: I mean I'm not doing the budget, but did you already booked this?

2024-08-15 13:05 sourabhniyogi: I did [50%]

2024-08-15 13:06 bkchr: So you mean you already paid 50% upfront?

2024-08-15 13:08 emielvanderhoek: I would be great to have some center of gravity mid November. Obviously there is some risk for the organizing party if funding is required. It would be good to get some consensus on date (and later content) for an event planning wise.

2024-08-15 13:08 sourabhniyogi: image.png

2024-08-15 13:09 sourabhniyogi: Approximately, yes. There is a OneBlock Polkadot hackathon where I'm not sure where they ended up.

2024-08-15 13:10 sourabhniyogi: They originally wanted 13th+14th so since it opened up I asked for it.

2024-08-15 13:11 emielvanderhoek: My JAM team (3x) has devcon tickets already. We have not yet booked flight and accommodation. Nice if we can get some reimbursement but not strictly necessary.

2024-08-15 13:11 rektormoon: So from the WebZero side I would like to involve the teams building on JAM to be on the sub0 program on the 11th.

2024-08-15 13:11 rektormoon: > <@sourabhniyogi:matrix.org> They originally wanted 13th+14th so since it opened up I asked for it. Overlaps with devcon, not wise

2024-08-15 13:13 emielvanderhoek: We’ll travel to where center of gravity is for JAM0 if any.

2024-08-15 13:13 gav: > <@sourabhniyogi:matrix.org> To state the obvious, a sub0 without you is barely a sub0 =) Well it certainly wasn’t my preference to run it twice in Bangkok in the same year.

2024-08-15 13:14 rektormoon: We felt the opportunity is too good to pass up 😅

2024-08-15 13:23 sourabhniyogi: Ok rektormoon 11th being the JAM day of sub0 makes sense, thanks for jumping in here -- I'll check if I can swap out the Valia room on the 11th for the 13th / 14th. We really want more time to work together in person than just 1 day.

2024-08-15 13:24 rektormoon: We plan to have a hacker space in sub0, so you could start work together there on the 9th and 10th

2024-08-15 13:24 rektormoon: Also open at night

2024-08-15 14:01 bkchr: > <@sourabhniyogi:matrix.org> Ok rektormoon 11th being the JAM day of sub0 makes sense, thanks for jumping in here -- I'll check if I can swap out the Valia room on the 11th for the 13th / 14th. We really want more time to work together in person than just 1 day. It would also be nice to be more cost efficient. Aka only one event. Can you cancel it? Or at least partly? People will also go to devcon, not sure that much will then appear there to code after a long day of conference

2024-08-15 14:11 sourabhniyogi: Alright updated https://polkadot.polkassembly.io/referenda/1024 -- its cost-efficient in that the cost is $0. I don't want to let anyone down.

2024-08-17 06:47 xlchen: While it is great if we can get a multi client testnet running on JAM0/Sub0 RESET, it is a relatively involved work and I am sure the first try wouldn't work. And I can already foresee some struggles that we are going to have due to number of components required for the whole thing to work especially many of the components are still in draft stage (e.g. networking). In simple words, this is too much for an initial MVP and there will be a good chance we won't be able to deliver it as there are too much uncertainties. It is a good target for us to try, but I won't bet for it. When a task is too big, what we do is to break it down. So here is my suggestion: Figure out a simple way for implementations to communicate. It needs to be simple, so pipes (i.e. stdout/stdin). Every program can print and shouldn't take more than 10 line of code to allow it to read. In that way we can have many standalone binaries confirming a same interface and we can assemble all those binaries together for lot more interesting tests. For example, team A have implemented all the components and team B only implemented Safrole, team C only implemented PVM. We can replace the Safrole@TeamA to Safrole@TeamB and PVM@TeamA to PVM@TeamC and still be able to setup a multi client testnet covers some work from team B and C. This also forces everyone to write modular code, which is a good thing as well. and this will be the first step https://github.com/w3f/jamtestvectors/issues/7

2024-08-17 11:32 sourabhniyogi: Awesome Bryan Chen | Laminar & Acala , thank you for suggesting it. The "pipes" model standing in place of QUIC and the binary from teams being the core deliverable would be immensely valuable -- is repurposing a fraction of ZombieNet \[which works using chain binaries\] useful for this? Its immediately clear this pipes+binaries approach is clearly a LOT easier for teams working together with as a proxy for JAMNP, automating the ip/port "setup", but probably can continue into JAMNP with "binaries trading" with a few flags switching modes. Having a test runner for the existing test vector set makes sense, but the PVM test vector case is quite a different character than all the others in that its almost totally removed from networking -- to use your concrete example, if "team C" only implemented PVM, they would only be able to run through the test runner on just the PVM, and not be able to do anything with "team A" and "team B". In contrast, team A+B will talk Safrole and can continue onto the other 4 extrinsics, which are .. not very modular. Team C hasn't gotten very far, not to pass too much judgement here, but has a LOT of catching up to do. We do have a tiny (V=6, C=2) QUIC passing Ticket + Blocks (with JSON serialization/deserialization instead of SCALE encoding/decoding) and are proceeding along with our own little RS code (with W\_C x W\_S \<\< 4104, like 24) to get the other 4 extrinsics with DA over the next 4-6 weeks, going from "Team B" to "Team A". We believe deviating from GP Appendix H to support a tiny RS code is essential for multi-team MVP success and hope we can "pick some parameters" together matched to the "tiny" Safrole test vectors. We believe a battery of "trivial" jam work packages / services (refine+accumulate, then adding in the auth+transfer) that utilize the host functions are critical for education and enabling team As to keep going. (In the absence of this, we are currently hand assembling our own little sum of squares battery, replacing a byte here and there for host function calls). We imagine the first A teams that publish 4-5 epochs (48-60 blocks) covering this battery can argue between themselves about state roots, but it would be even better if we had a authoritative "M0" 48-60 block dump that covered all the extrinsics, modeled DA, to make arguments tend to zero. Then we all would just try to get M0 to work on our own first as a prereq to binary trading. We would be happy to supply whatever binary works by early Oct and hope there are a bunch of "A teams" and "B teams" where we can do pre-work in October. If there are any A candidates who get to pipes model before Oct, we'll be inspired to match you, pls share! (edited)

2024-08-17 12:42 gav: > <@sourabhniyogi:matrix.org> Awesome Bryan Chen | Laminar & Acala , thank you for suggesting it. The "pipes" model standing in place of QUIC and the binary from teams being the core deliverable would be immensely valuable -- is repurposing a fraction of ZombieNet \[which works using chain binaries\] useful for this? > > Its immediately clear this pipes+binaries approach is clearly a LOT easier for teams working together with as a proxy for JAMNP, automating the ip/port "setup", but probably can continue into JAMNP with "binaries trading" with a few flags switching modes. > > Having a test runner for the existing test vector set makes sense, but the PVM test vector case is quite a different character than all the others in that its almost totally removed from networking -- to use your concrete example, if "team C" only implemented PVM, they would only be able to run through the test runner on just the PVM, and not be able to do anything with "team A" and "team B". In contrast, team A+B will talk Safrole and can continue onto the other 4 extrinsics, which are .. not very modular. Team C hasn't gotten very far, not to pass too much judgement here, but has a LOT of catching up to do. > > We do have a tiny (V=6, C=2) QUIC passing Ticket + Blocks (with JSON serialization/deserialization instead of SCALE encoding/decoding) and are proceeding along with our own little RS code (with W\_C x W\_S \<\< 4104, like 24) to get the other 4 extrinsics with DA over the next 4-6 weeks, going from "Team B" to "Team A". We believe deviating from GP Appendix H to support a tiny RS code is essential for multi-team MVP success and hope we can "pick some parameters" together matched to the "tiny" Safrole test vectors. > > We believe a battery of "trivial" jam work packages / services (refine+accumulate, then adding in the auth+transfer) that utilize the host functions are critical for education and enabling team As to keep going. (In the absence of this, we are currently hand assembling our own little sum of squares battery, replacing a byte here and there for host function calls). We imagine the first A teams that publish 4-5 epochs (48-60 blocks) covering this battery can argue between themselves about state roots, but it would be even better if we had a authoritative "M0" 48-60 block dump that covered all the extrinsics, modeled DA, to make arguments tend to zero. Then we all would just try to get M0 to work on our own first as a prereq to binary trading. > > We would be happy to supply whatever binary works by early Oct and hope there are a bunch of "A teams" and "B teams" where we can do pre-work in October. If there are any A candidates who get to pipes model before Oct, we'll be inspired to match you, pls share! Indeed, such a dump is in line with my expectations of what we’ll provide for M1 tests.

2024-08-18 16:28 shwchg: davxy: Hi, I have a question about your dispute test vector You mentioned in your README that when an execution error occurs, the post-state must match the pre-state. But some of your vectors are still being updated. https://github.com/davxy/jam-test-vectors/blob/disputes/disputes/tiny/progress_with_culprits-3.json

"pre_state": {
        "psi": {
            "psi_g": [],
            "psi_b": [],
            "psi_w": [],
            "psi_o": []
        },
        "rho": [
            null,
            null
        ]
"post_state": {
        "psi": {
            "psi_g": [],
            "psi_b": [
                "0x11da6d1f761ddf9bdb4c9d6e5303ebd41f61858d0a5647a1a7bfe089bf921be9"
            ],
            "psi_w": [],
            "psi_o": []
        },
        "rho": [
            null,
            null
        ]

So are there specific errors that you handle in a way that allows the process to continue running?

2024-08-18 16:32 davxy: Nice catch. I'll fix it. For what concerns test vectors, on error post matches pre

2024-08-18 16:40 shwchg: ok thanks!

2024-08-18 17:01 davxy: Fixed. Thank you for reporting :-)

2024-08-18 19:34 gav: This is not the place to argue over treasury expenditure. Go to Polkassembly or wherever to do that and vote with your stake.

2024-08-18 19:37 gav: @sourabhniyogi:matrix.org Please remove the offending text and be more careful in the future.

2024-08-20 20:14 danicuki: I am organizing an informal JAM enthusiasts meetup at Lisbon, for those who are nearby: https://lu.ma/fqg0ktn0

2024-08-20 20:41 danicuki: For those who are building JAM, I suggest adding your org / repo to the Electric Capital Crypto Report repo: https://github.com/electric-capital/crypto-ecosystems Here is a PR reference to follow: https://github.com/electric-capital/crypto-ecosystems/pull/1443

2024-08-20 20:57 vinsystems: > <@danicuki:matrix.org> I am organizing an informal JAM enthusiasts meetup at Lisbon, for those who are nearby: https://lu.ma/fqg0ktn0 We are from Spain but we won't be able to attend at that date

2024-08-21 05:59 xlchen: I am refactoring our codec (i.e. ditch the existing ScaleCodec library) and just realize that was the Safrole test vector using Scale codec instead of Jam codec? davxy

2024-08-21 06:00 xlchen: the length prefix in the provided tests are encoded using compact encoding, not the Jam natural number serialization method

2024-08-21 06:19 luka99: Regarding availability using erasure-coding, it says in 14.2.2 that the justification data needed to verify the segments/chunks is too large to be stored by every validator and that it too goes through the same availability framework. Doesn't that create the same problem as to why the justification data is needed in the first place? Someone can give a corrupted chunk of the justification data and we can't restore it?

2024-08-21 06:27 davxy: Current vectors are provided in two flavors. SCALE and JSON. (edited)

2024-08-21 06:28 davxy: Indeed at the beginning only json was provided. But IIRC someone asked for a scale version as well (edited)

2024-08-21 06:30 davxy: I guess SCALE vectors will be probably replaced at some point

2024-08-21 06:45 xlchen: it was me asking for it with assumption that scale is used for jam, but that’s actually not the case so will need a version for Jam codec

2024-08-21 06:46 xlchen: I think the only difference is just the encoding format for compact number

2024-08-21 09:11 tomusdrw: Hi all! Our team was recently working on a PVM implementation and we've also built a tool that we hope will help other implementors. https://pvm.fluffylabs.dev/ <-- here you can find our PVM Disassembler. It can load a JSON test case file and visualise what instructions are used in the code. By plugging a PVM implementation you can also turn it into a debugger. If any of the teams is interested we want to make it possible to upload a WASM blob of your PVM implementation and run it instead of ours (see open issue on github regarding the interface). We plan to support running two PVMs in parallel and checking their state (registers / memory / gas) at every step and display the exact instruction when they start to differ. Tool's code is open source on github, so feel free to fork or report any issues or feature requests there. We are looking forward for your feedback.

2024-08-21 09:11 tomusdrw: Huge kudos for Wojciech Kwiatek Krystian Fras who are behind the code of this one :) 👏👏👏

2024-08-21 15:10 rmagon: Hi - for those who are implementing in Go, are there any native libraries for cryptographic primitives like the ones in Rust? For, say, Bandersnatch, RingVRF etc. ? I can see that it is allowed to use such libs but couldn't find anything that we could rely on, specially in Go. Or are you implementing them from scratch Or using FFI for M1 atleast?

2024-08-21 15:23 rick: I don’t think there are specific implementations but Filipo Valsorda has published a few of the necessary lower level primitives. I’ll post here if I find more specific ones. Other people to check out are probably David Stainton from katzenpost, Yawning, and I will probably post mine as a separate oss lib if I get to it. (edited)

2024-08-21 15:24 subotic: > <@rmagon:matrix.org> Hi - for those who are implementing in Go, are there any native libraries for cryptographic primitives like the ones in Rust? For, say, Bandersnatch, RingVRF etc. ? I can see that it is allowed to use such libs but couldn't find anything that we could rely on, specially in Go. Or are you implementing them from scratch Or using FFI for M1 atleast? FFI is explicitly allowed in these cases (for all milestones).

2024-08-22 09:40 luke_f: image.png

2024-08-22 09:40 luke_f: @davxy:matrix.org looking at bandersnatch-vrf code example here https://github.com/davxy/bandersnatch-vrfs-spec/blob/ae82b19e99cdc1301820d9cbd978e0a73b9de2a8/example/src/main.rs#L52 i see that Prover has prover_idx field I'm assuming this corresponds to the index into the validator set of said signer/ prover could you confirm please?@davxy:matrix.org (edited)

2024-08-22 09:45 davxy: Confirmed

2024-08-22 09:45 luke_f: Thank you :)

2024-08-23 12:33 rmagon: > <@subotic:matrix.org> FFI is explicitly allowed in these cases (for all milestones). Makes sense! thank you!

2024-08-23 12:33 rmagon: > <@rick:carback.us> I don’t think there are specific implementations but Filipo Valsorda has published a few of the necessary lower level primitives. > > I’ll post here if I find more specific ones. Other people to check out are probably David Stainton from katzenpost, Yawning, and I will probably post mine as a separate oss lib if I get to it. That would be great if you can find anything. I will do the same :)

2024-08-23 18:43 sourabhniyogi: sub0 with a bit of JAM is up for vote at https://polkadot.polkassembly.io/referenda/1113 -- This is followed by 5 days of "JAM0" for Implementers (venue sponsored by us, let me know if you are JAM implementers and want in =). If you intend to trek to sub0 in Bangkok and Devcon but haven't gotten tickets, here is where you can get the final wave on September 3: https://devcon.org/en/tickets/ If you want discounted hotel rooms use "JAM0" at https://valiahotelbangkok.com (edited)

2024-08-23 19:41 emielvanderhoek: There was discussion on a Bangkok & Lisbon JAM0 gathering. With the Sub0 addition is it now all happening in Bangkok? (edited)

2024-08-24 09:29 bmaas: > <@sourabhniyogi:matrix.org> sub0 with a bit of JAM is up for vote at https://polkadot.polkassembly.io/referenda/1113 -- This is followed by 5 days of "JAM0" for Implementers (venue sponsored by us, let me know if you are JAM implementers and want in =). If you intend to trek to sub0 in Bangkok and Devcon but haven't gotten tickets, here is where you can get the final wave on September 3: > https://devcon.org/en/tickets/ > If you want discounted hotel rooms use "JAM0" at https://valiahotelbangkok.com That sounds like a brilliant idea. I will definitely be up for it.

2024-08-24 17:20 prematurata: Hello all i'm trying to implement the verification for the ticket extrinsic (74), I am using the data coming from https://github.com/w3f/jamtestvectors/blob/master/safrole/tiny/publish-tickets-no-mark-2.json so basically this inputs: - so signature from input.extrinsic[0].signature = 0xb342bf8f...85a8 - and ringroot commitment pre_state.gamma_z = 0x96a4b4...74cf - posterior eta_2 from post_state.eta[2] = 0xbb30..14aa according to paper i just need to concatenate $jam_ticket_seal with posterior eta 2 and the attempt (1 in the json) this (if i understand properly) should be the input data. I am not able to get the code to verify the signature. I basically checked this piece of code for it https://github.com/davxy/bandersnatch-vrfs-spec/blob/ae82b19e99cdc1301820d9cbd978e0a73b9de2a8/example/src/main.rs#L134 and modified to have the ring context to be of size 6 and pass an empty aux_data... but Public::verify().is_err() is always true I am a bit lost to be honest any help would be appreciated.

2024-08-24 17:42 gav: > <@prematurata:matrix.org> Hello all i'm trying to implement the verification for the ticket extrinsic (74), > > I am using the data coming from https://github.com/w3f/jamtestvectors/blob/master/safrole/tiny/publish-tickets-no-mark-2.json > > so basically this inputs: > - so signature from input.extrinsic[0].signature = 0xb342bf8f...85a8 > - and ringroot commitment pre_state.gamma_z = 0x96a4b4...74cf > - posterior eta_2 from post_state.eta[2] = 0xbb30..14aa > > according to paper i just need to concatenate $jam_ticket_seal with posterior eta 2 and the attempt (1 in the json) > > this (if i understand properly) should be the input data. > > I am not able to get the code to verify the signature. I basically checked this piece of code for it > https://github.com/davxy/bandersnatch-vrfs-spec/blob/ae82b19e99cdc1301820d9cbd978e0a73b9de2a8/example/src/main.rs#L134 > > and modified to have the ring context to be of size 6 and pass an empty aux_data... but Public::verify().is_err() is always true > > I am a bit lost to be honest any help would be appreciated. One for davxy to look at.

2024-08-24 17:49 motifnetwork: > <@emielvanderhoek:matrix.org> There was discussion on a Bangkok & Lisbon JAM0 gathering. With the Sub0 addition is it now all happening in Bangkok? Hear the same at the AAG

2024-08-24 17:49 motifnetwork: yep, here https://polkadot.polkassembly.io/referenda/1113

2024-08-25 19:58 davxy: prematurata: https://gist.github.com/davxy/54892e1ebd548040615a9fdc627de632 I hope it helps

2024-08-26 10:46 davxy: Another batch of test vectors: https://github.com/w3f/jamtestvectors/pull/11 This one focuses on the "Recent History" STF (section 7). Overall, it's fairly straightforward, but the update to MMR peaks adds some interesting elements. Any feedback or notes are appreciated. Thanks!

2024-08-27 07:09 ksc85pwpj5:

2024-08-27 07:09 ksc85pwpj5:

2024-08-27 07:09 ksc85pwpj5:

2024-08-27 07:12 ksc85pwpj5:

2024-08-27 07:12 ksc85pwpj5:

2024-08-27 07:12 ksc85pwpj5:

2024-08-27 09:06 niubob: Hi, I just saw Gav's notes, where can I see the current progress of each team? Or are there any language sets that no one is participating in?

2024-08-28 18:36 sourabhniyogi:

2024-08-29 03:33 xlchen: Can someone start merging PRs at https://github.com/w3f/jamtestvectors? Otherwise we have to use our own forks to be able to run all the tests on different branches and they are start getting merge conflicts now

2024-08-29 14:02 ntnntn: Is there any rules regarding using FFI for stuff that is implemented in-house? E.g. we run for a TS implementation, can we offload any of the lower-level stuff that require performance to Rust components we would also write from scratch and expose them via FFI?

2024-08-29 14:02 ntnntn: Or does FFI only apply for the existing libraries that are allowed to be used?

2024-08-29 18:42 davxy: I'm working on a version of the vectors using the codec prescribed by GP. It's almost ready. So maybe it's worth waiting? I'll submit the vectors for the codec first (edited)

2024-08-30 16:37 davxy: https://github.com/w3f/jamtestvectors/pull/12 Your feedback is appreciated

2024-08-30 17:01 philip.poloczek:

2024-08-30 17:01 sourabhniyogi: This is awesome and gives us a lot to chew on, thank you! All the components of Eq (289), C(1)-C(13) and at least 1 test case for service state (the last 4) would be awesome, culminating in state root that we can strive for. Generally having a codectest for all the JAMNP objects would be useful as well, most of which are inside your test vectors already, but not all. Having network-centric test vectors would be awesome alongside this "block" centric view. We are wondering if Vote would be broadcasted by validators instead of Dispute objects in JAMNP. (edited)

2024-08-30 20:03 dave:

2024-08-30 23:31 sourabhniyogi: https://github.com/davxy/jam-test-vectors/blob/codec-vectors/codec/data/work_package.json#L36-L45

2024-08-30 23:31 sourabhniyogi: image.png

2024-08-30 23:32 sourabhniyogi: Issue: We were expecting arbitrary length blobs in the "extrinsics" attribute of a Work Package, not (hash, length) combinations. Section 14.3 says "In general this data will be passed to the guarantor alongside the work-package." I guess the "alongside" doesn't mean "inside" here but then we're left wondering how these work packages could get _audited_ if only the guarantors get the extrinsics "alongside" the work package rather than inside the work package. (edited)

2024-08-31 06:56 davxy:

2024-08-31 06:57 davxy: That doesn't seem to be what the gray paper suggests, see eq 285 (GP 0.3.6)

2024-08-31 16:57 sourabhniyogi: davxy After review we don't have an issue =) We need a fully worked out case (in your style =)) of the Availability Specifier inputs ${\bf b}$ and ${\bf s}$, intermediate Encode(p, x, i, j), where p is the package, x is the [hash, len], i is the imported data, and j the justifications for this imported data -- culminating in a JAMP instruction for sharing (justified) DA chunks, which gav says will likely just be Vec<Hash> (blob subtree) ++ Blob ++ Vec<Hash> (segments subtree) ++ Vec<SegmentChunk> ++ Vec<Hash> (super-tree) and maps onto these Encoded inputs. It is super helpful to have you choose verbose JSON attribute names both consistent with everything else and disambiguating. Even better would be a diagram though =)

2024-09-02 10:22 rhee: kianenigma: Hey Kian, this is the first Korean_translated version of your JAM summary blog. I've invited a professional local technical writer + one of the research firms to deliver your contents in a correct tone & manner. Hope you like it and link to your blog article: https://research.despread.io/ko/report-jam/

2024-09-04 07:54 vinsystems: Where do you guys define the PVM's ram memory: in the stack or in the heap? The max ram size of a 32bit proccessor is 4GB, so defining it in the stack may be better in terms of performance, but will also consume a lot of system resources. (edited)

2024-09-04 08:12 vinsystems: Where do you guys have define the PVM's ram memory: in the stack or in the heap? The max ram size of a 32bit proccessor is 4GB, so defining it in the stack may be better in terms of performance but will also consume a lot of system resources.

2024-09-04 08:16 dakkk: > <@vinsystems:matrix.org> Where do you guys have define the PVM's ram memory: in the stack or in the heap? The max ram size of a 32bit proccessor is 4GB, so defining it in the stack may be better in terms of performance but will also consume a lot of system resources. I'm currently using the heap

2024-09-04 08:16 jan: 1. Competently implemented there is no performance difference. 2. No one sane ever puts something as big as 4GB on the stack, nevermind that by default it's not even possible since every OS has a strict stack limit that is at most a few megabytes (it can be increased, but the idea of putting a 4GB value on the stack is so insane that I don't even know if you can increase it that much, as no on ever does this). (edited)

2024-09-09 07:09 gav: > <@sourabhniyogi:matrix.org> Issue: We were expecting arbitrary length blobs in the "extrinsics" attribute of a Work Package, not (hash, length) combinations. Section 14.3 says "In general this data will be passed to the guarantor alongside the work-package." I guess the "alongside" doesn't mean "inside" here but then we're left wondering how these work packages could get _audited_ if only the guarantors get the extrinsics "alongside" the work package rather than inside the work package. A Work Package contains only (hash, length) pairs to define extrinsic data. Guarantors may disregard a Work Package if they have been failed to be furnished with the correct preimages (it's up to the submitter to give them the real extrinsic data in addition to the WP).

2024-09-09 07:10 gav: As for auditors, extrinsic data is contained in the Work Package Bundle, which is made available.

2024-09-09 12:25 davxy: Could I kindly request concise feedback from the various teams regarding this PR: https://github.com/w3f/jamtestvectors/pull/12? If you encounter any issues, please report them directly in the PR.

2024-09-09 12:26 davxy: Please give a thumbs up if you process it correctly, and a thumbs down if you encounter any issues.

2024-09-09 13:05 emielvanderhoek:

2024-09-09 13:09 emielvanderhoek: Added a comment in the Safrole PR for one more Safrole testvector.

2024-09-11 08:50 luke_f:

2024-09-11 08:55 luke_f: Screenshot from 2024-09-11 15-50-26.png

2024-09-11 08:55 luke_f: davxy: question for you sir: I have a problem with decoding "blank" public key into an AffinePoint when creating epoch root / ringCommitment in the attached image, we see that we create a new epoch root with possibly some "blank" public bandersnatch keys when i try to decode these into an AffinePoint using point_decode https://github.com/davxy/ark-ec-vrfs/blob/main/src/codec.rs the decoding fails could you suggest please how to handle these (edited)

2024-09-11 09:15 gav: > <@luke_f:matrix.org> davxy: question for you sir: > I have a problem with decoding "blank" public key into an AffinePoint when creating epoch root / ringCommitment > > in the attached image, we see that we create a new epoch root with possibly some "blank" public bandersnatch keys > > when i try to decode these into an AffinePoint using point_decode https://github.com/davxy/ark-ec-vrfs/blob/main/src/codec.rs > the decoding fails > could you suggest please how to handle these Don't decode them.

2024-09-11 09:15 gav: https://www.goodreads.com/quotes/7570350-the-patient-says-doctor-it-hurts-when-i-do-this

2024-09-11 09:19 luke_f: right, but then the size of the ring would change. by your answer i gather that's ok edit: what i mean is, if I'm not decoding them back to a point, then i understand i just filter these keys out when creating a new epoch root, but the GP does not mention any filtering action here, so I'm not sure how to procceed (edited)

2024-09-11 09:26 gav: From the GP perspective, there is no filtering action.

2024-09-11 09:26 gav: However, said filtering should be defined within the Bandersnatch appendix. I introduced a note Bandersnatch: Ignore bad keys

2024-09-11 09:27 gav: In general bad keys should just be noops (in the case of adding to a ring) or fail (in the case of signature verification). The GP doesn't care about specific representations of the key. From its perspective keys are just opaque 32-byte blobs. It's only within the Bandersnatch def that we attempt to make sense of the blob, and thus it is there where the meaning of "bad" keys is defined. (edited)

2024-09-11 09:28 gav: (This is not limited to the nullifier key either: we don't prevent validators from uploading bad keys.) (edited)

2024-09-11 09:30 luke_f: understood. Thank you gav

2024-09-11 10:25 davxy: luke_f: Your question reminded me of an interesting missing case in the test vectors, which I had noted earlier here: https://github.com/w3f/jamtestvectors/pull/8#issuecomment-2275460028. Thank you for the reminder! I will proceed with adding it. :-)

2024-09-11 10:26 luke_f: > <@davxy:matrix.org> luke_f: Your question reminded me of an interesting missing case in the test vectors, which I had noted earlier here: https://github.com/w3f/jamtestvectors/pull/8#issuecomment-2275460028. > Thank you for the reminder! I will proceed with adding it. :-) my pleasure :)

2024-09-12 03:01 gav: > <@gav:polkadot.io> However, said filtering should be defined within the Bandersnatch appendix. I introduced a note Bandersnatch: Ignore bad keys Note: After davxy's comment, the GP appendix was updated to state that when building the ring, do not skip but instead use the Bandersnatch padding point for any keys with no point of their own.

2024-09-12 13:17 prematurata: question about the host call definition. Appendix A.6. it looks like that the invocation shouldreturn x' when a page fault happen when callling f but x' is not defined anywhere so i guess it's just a typo and instead of x' we want x... is this assumption correct? Also I have a couple of questions about gammax (green underline). - my understanding is that since it should be a type any of the Accumulate functions can be expected there. is it correct? - also not sure its intended but there are 2 gamma x definitions. one in accumulate and one in general functions

2024-09-12 13:17 prematurata: image.png

2024-09-12 15:07 jimboj21: Hey davxy I saw in the codec PR that you are keeping tabs on teams that are able to process the codec test vectors, so wanted to say we (Gossamer team) are able to fully process them 👍️ Thanks for the work getting us the vectors!

2024-09-12 15:18 prematurata: I also think the B.2 is-authorized invocation ΨI has something strange. - ΨM is being called with 1st param p_c - p is of type workpackage ( P ) - _c is codehash - ΨM first param is a member of Yor generic blob but that is being used for Y() fn in 239 - 239 implies that if all the statements cannot be satisfied, then basically Y() output is "empty/nil" - this means that since the ΨI always passess a code hash then ΨM is always returning panic due to 246 (edited)

2024-09-12 15:46 gav: here it is p_c where the c is bold and not italic, thereby referring to the code preimage not the hash.

2024-09-12 15:50 gav: > <@prematurata:matrix.org> question about the host call definition. Appendix A.6. it looks like that the invocation shouldreturn x' when a page fault happen when callling f but x' is not defined anywhere so i guess it's just a typo and instead of x' we want x... is this assumption correct? > > > Also I have a couple of questions about gammax (green underline). > > - my understanding is that since it should be a type any of the Accumulate functions can be expected there. is it correct? > - also not sure its intended but there are 2 gamma x definitions. one in accumulate and one in general functions > Yes that should be x not x’. Will be fixed in next release.

2024-09-12 15:52 gav: @prematurata:matrix.orgthis stuff is OT in here. In future please use the gray paper channel. (edited)

2024-09-13 04:13 gav: > <@prematurata:matrix.org> question about the host call definition. Appendix A.6. it looks like that the invocation shouldreturn x' when a page fault happen when callling f but x' is not defined anywhere so i guess it's just a typo and instead of x' we want x... is this assumption correct? > > > Also I have a couple of questions about gammax (green underline). > > - my understanding is that since it should be a type any of the Accumulate functions can be expected there. is it correct? > - also not sure its intended but there are 2 gamma x definitions. one in accumulate and one in general functions prematurata: please explain what you mean by "gamma x".

2024-09-13 07:54 prematurata: > <@gav:polkadot.io> prematurata: please explain what you mean by "gamma x". sorry I meant omega

2024-09-13 08:12 gav: > <@prematurata:matrix.org> sorry I meant omega Fixed in https://github.com/gavofyork/graypaper/pull/77/commits/b1f172bdf98a52b12cebd9d5064b22230730b5d9

2024-09-13 08:13 prematurata: > <@gav:polkadot.io> Fixed in https://github.com/gavofyork/graypaper/pull/77/commits/b1f172bdf98a52b12cebd9d5064b22230730b5d9 movin discussion in graypaper chan

2024-09-13 10:41 bmaas: > <@davxy:matrix.org> Please give a thumbs up if you process it correctly, and a thumbs down if you encounter any issues. https://github.com/w3f/jamtestvectors/pull/12#issuecomment-2348625189 I have a question here davxy

2024-09-13 11:59 davxy: I reviewed your comments. The reasons are design decisions, so you should eventually raise your concerns in the graypaper. In any case, IMHO, it is better to maintain a uniform design for u16/u32 encoding in the protocol rather than saving 1 byte here and there by introducing pitfalls related to variable-length encoding.

2024-09-13 12:10 davxy: Side note for slot number. Even though the test vectors are using small slot numbers, in production the slot count begins with the "JAM common era" (1 Jan 2024 = 1704110400). As of today, you would already have a slot = (1726229014 - 1704110400) / 6 = 3686435 = 0x384023 (3 bytes). (edited)

2024-09-13 12:15 bmaas: image.png

2024-09-13 12:24 bmaas: Thanks for the prompt reply davxy . I'm not suggesting saving everything in just one byte. When I study the formula—272—I expect that the length of the encoding should be derivable from the l-value. Thus in the example header-0.json case, l would be 0, and thus following the formula it would produce 0 extra octets. For this reason I was not expecting the extra 3 bytes.

2024-09-13 12:29 bmaas: > <@davxy:matrix.org> Side note for slot number. Even though the test vectors are using small slot numbers, in production the slot count begins with the "JAM common era" (1 Jan 2024 = 1704110400). As of today, you would already have a slot = (1726229014 - 1704110400) / 6 = 3686435 = 0x384023 (3 bytes). I believe I found the misunderstanding, 272 , there is a note: at present utilized only in encoding the length prefix of variable-length sequences.

2024-09-13 12:41 davxy: Screenshot from 2024-09-13 14-41-11.png

2024-09-13 12:42 davxy: Also H_t is explicitly encoded as E_4 , so you need to refer to the formula for fixed length integers (just above the one you're looking at)

2024-09-16 07:53 xlchen: gav: there are many open PR here https://github.com/w3f/jamtestvectors/pulls can we have the ready one merged? i.e. #2 and #12

2024-09-18 01:12 gav: Merged

2024-09-18 08:13 stephenta: Hi Gavin and fellowship, Our team has an interest in working on JAM implementation. Can I ask a question is there any criteria of a success deliverable milestone?

2024-09-19 08:41 luke_f: davxy: apologies in advance for the kinda silly question: is there ANY CHANCE that the order of gamma_s in tiny/enact-epoch-change-with-no-tickets-4.json is wrong? asking because I'm getting different order 😅

2024-09-20 03:11 xlchen: Some question about Safrole ticket submission. https://hackmd.io/@polkadot/jamsnp To confirm my understanding first: - On each epoch, every next epoch validator (γk) can submit to up two tickets - Block authors are incentivized to include up to 16 tickets in extrinsic - Potential block authors should maintain an tickets pool and include top tickets when authoring new blocks - Tickets are not gossiped. So every next epoch validators should submit their two tickets to every current epoch validators. - Next epoch validators shouldn't need to resubmit their tickets or it could be counted as spam? - So current validators should persist the pending tickets pool across restarts - In order to make it possible for the first block of an epoch to include full 16 tickets, next next epoch validators (ι) should start generating and submit tickets before they become γk. The ring commitment will depends on ψo but it is unlikely for it to change at and of an epoch as we are not expecting punishments in more than 99.9% of the time. - So validators should be expected to receive tickets for both current and next ring commitment And my questions: As JAMSNP currently defined, ticket submission seems like is only for 1 ticket only? But validator are almost always want to send two tickets. So better to make it to expect two tickets What exactly does the success / bad mean? Ack the data is received or expecting the receiver to validate the data? i.e. verify it is a valid ticket and not a duplicated etc. I will image it will be just ack but need to confirm. What is the expected optimal behaviour of ticket submission? Every next next epoch validators (ι) submit tickets at and of an epoch and resubmits at start of an epoch in case the expected ring commitment changed? So validators are expected to receive tickets for two different epoch. In that case, should the ticket submission protocol to include an epoch index to keep things simpler? Otherwise the receiver may need to verify the tickets twice to figure out if it is valid or not. (edited)

2024-09-20 07:53 gav: > <@xlchen:matrix.org> Some question about Safrole ticket submission. https://hackmd.io/@polkadot/jamsnp > To confirm my understanding first: > > - On each epoch, every next epoch validator (γk) can submit to up two tickets > - Block authors are incentivized to include up to 16 tickets in extrinsic > - Potential block authors should maintain an tickets pool and include top tickets when authoring new blocks > - Tickets are not gossiped. So every next epoch validators should submit their two tickets to every current epoch validators. > - Next epoch validators shouldn't need to resubmit their tickets or it could be counted as spam? > - So current validators should persist the pending tickets pool across restarts > - In order to make it possible for the first block of an epoch to include full 16 tickets, next next epoch validators (ι) should start generating and submit tickets before they become γk. The ring commitment will depends on ψo but it is unlikely for it to change at and of an epoch as we are not expecting punishments in more than 99.9% of the time. > - So validators should be expected to receive tickets for both current and next ring commitment > > And my questions: > As JAMSNP currently defined, ticket submission seems like is only for 1 ticket only? But validator are almost always want to send two tickets. So better to make it to expect two tickets > What exactly does the success / bad mean? Ack the data is received or expecting the receiver to validate the data? i.e. verify it is a valid ticket and not a duplicated etc. I will image it will be just ack but need to confirm. > What is the expected optimal behaviour of ticket submission? Every next next epoch validators (ι) submit tickets at and of an epoch and resubmits at start of an epoch in case the expected ring commitment changed? > So validators are expected to receive tickets for two different epoch. In that case, should the ticket submission protocol to include an epoch index to keep things simpler? Otherwise the receiver may need to verify the tickets twice to figure out if it is valid or not. davxy: ^^

2024-09-20 08:26 dave: I've got some pending changes to JAMSNP. For Safrole ticket submission I currently have this... ### CE 128/129: Safrole ticket submission Sharing of a Safrole ticket for inclusion in a block. Safrole tickets are distributed in the epoch prior to the one in which they are to be used. They are distributed in two steps. Each ticket is first sent from the generating validator to a deterministically-selected "proxy" validator. This proxy validator then sends the ticket to _all_ current validators. The first step should be performed shortly after the connectivity changes for a new epoch are applied. The index of the proxy validator for a ticket is determined by interpreting the last 4 bytes of the ticket's VRF output as a big-endian unsigned integer, modulo the number of validators minus 1. The proxy validator is selected from the next epoch's validator list, excluding the generating validator. Proxy validators should verify the signature of any ticket they receive, and verify that they are the correct proxy for the ticket. If these checks succeed, they should forward the ticket to all current validators. Forwarding should be delayed until a fixed point in the epoch (eg 3 minutes in), to avoid exposing the timing of the initial message. Forwarding should be spaced out over the first half of the Safrole lottery period. Forwarding may be stopped if the ticket is included in a block. Protocol 128 is used for the first step (generating validator to proxy validator), protocol 129 is used for the second step (proxy validator to all current validators).

Attempt = { 0, 1 }
Ticket = Attempt ++ Signature

Validator -> Validator

--> Ticket
--> FIN
<-- FIN

2024-09-20 08:32 xlchen: when does the connectivity change applies? it must happen before the new epoch starts right?

2024-09-20 08:32 dave: > As JAMSNP currently defined, ticket submission seems like is only for 1 ticket only? But > validator are almost always want to send two tickets. So better to make it to expect two tickets Might be changed in the full protocol but for simplicity we decided to stick with a single ticket for JAMSNP. > What exactly does the success / bad mean? Ack the data is received or expecting the receiver to validate the data? i.e. verify it is a valid ticket and not a duplicated etc. I will image it will be just ack but need to confirm. I've taken out success/bad. Failure should be indicated by resetting/stopping the stream. I think for Safrole you can finish (= succeed) as soon as you have received the data, ie just ack. > What is the expected optimal behaviour of ticket submission? Every next next epoch validators (ι) > submit tickets at and of an epoch and resubmits at start of an epoch in case the expected ring > commitment changed? I think this is described now. > So validators are expected to receive tickets for two different epoch. In that case, should the > ticket submission protocol to include an epoch index to keep things simpler? Otherwise the > receiver may need to verify the tickets twice to figure out if it is valid or not. Validators should only expect tickets for the next epoch.

2024-09-20 08:33 dave: > <@xlchen:matrix.org> when does the connectivity change applies? it must happen before the new epoch starts right? No. I currently have this defined as follows... ### Epoch transitions At the beginning of a new epoch, validators should wait to apply connectivity changes until both: - The first block in the epoch has been finalized. - At least 2 minutes have elapsed since the beginning of the epoch. This rule is intended to synchronize connectivity changes across validators, making epoch transitions smoother.

2024-09-20 08:34 dave: FWIW this is still all subject to change...

2024-09-20 08:39 xlchen: i see. I will assume all tickets are for current epoch

2024-09-20 08:41 dave: To be included in current epoch blocks, yeah. I don't think you should get tickets for other epochs unless a node is really confused 😅

2024-09-20 09:12 davxy: > <@luke_f:matrix.org> davxy: apologies in advance for the kinda silly question: > is there ANY CHANCE that the order of gamma_s in tiny/enact-epoch-change-with-no-tickets-4.json is wrong? > > asking because I'm getting different order 😅 @luke_f:matrix.org I'll take a look. I'll be afk until tomorrow 😅

2024-09-20 12:56 danicuki: I am organizing a JAM meetup in Lisbon on Oct 21th right after ETH Lisbon. The meetup will be at Parity's office and everyone is invited to join. https://lu.ma/fqg0ktn0 We are looking for sponsors for foods and drinks. DM me if you are interested

2024-09-21 02:54 gav: > <@danicuki:matrix.org> I am organizing a JAM meetup in Lisbon on Oct 21th right after ETH Lisbon. The meetup will be at Parity's office and everyone is invited to join. > > https://lu.ma/fqg0ktn0 > > We are looking for sponsors for foods and drinks. DM me if you are interested > I’ll probably stop by.

2024-09-23 22:11 celadari: Regarding ticket submission for Safrole, would it make sense to adopt an onion routing approach? Not for initial release but for latter. Each ticket could be sent through 3 different routes, with 3 proxies per route. The message would be encrypted for each proxy along the way, and only the final proxy decrypts the ticket without knowing the original sender. We can deterministically choose the routes using VRF output for different inputs and require a ring root signature for validation.

2024-09-23 22:49 gav: This is indeed the plan. Perhaps davxy wants to comment further.

2024-09-24 09:59 davxy: The idea has been considered and maybe, in some variant, will become part of the final protocol for ticket submission. Initially, however, the protocol described by David Emett (JAMSNP CE 128/129) will be adopted, where the ticket is submitted via a proxy whose identity is pseudo-randomly determined by the ticket ID. This means that anyone can infer the proxy responsible for a given ticket T (i.e., the proxy is f(T) and he knows who is the ticket creator). In the upcoming weeks, various possibilities will certainly be discussed. However, it is important to understand whether the existence of this "public" actor (aka proxy) who knows the identity of the ticket author is required, for instance, to efficiently submit information directly to the next block author directly via the proxy. Honestly this point is not 100% clear to me yet. If the presence of a public proxy to the ticket author is indeed required, the relevance of onion routing becomes a bit less relevant. Although it may still useful for some other aspects, such make traffic analysis more complicated. Proxy can still exist even with onion routing, e.g. by disclosing ticket's author identity only to the exit node and reusing the idea of CE 128 for deterministic proxy in function of T. I would be happy to hear more ideas in this area, indeed!

2024-09-24 15:14 danicuki: > <@danicuki:matrix.org> I am organizing a JAM meetup in Lisbon on Oct 21th right after ETH Lisbon. The meetup will be at Parity's office and everyone is invited to join. > > https://lu.ma/fqg0ktn0 > > We are looking for sponsors for foods and drinks. DM me if you are interested I am preparing the agenda for the JAM meetup Oct 21st. If anyone has a proposal for lightning talk / panel, workshop or whatever, DM me.

2024-09-24 17:42 davxy: I have updated the outstanding test vectors PRs (safrole, history, disputes) to include vectors encoded using the new compact integer codec as prescribed by the GP. - The vectors using the new codec have the extension *.bin. The old *.scale files are temporarily maintained in case anyone is still working with the old codec. These will be removed in the near future. - Safrole vectors have also been updated in content. The ring-proof backend introduced some internal changes which invalidate the old signatures (and tickets). If you are using ark-ec-vrfs, please update to the latest revision, as ticket verification will fail otherwise. - ark-ec-vrfs now provides a function to retrieve the padding point to be used to replace invalid keys when computing the ring commitment (https://github.com/davxy/ark-ec-vrfs/blob/e491b41b48d9132f59ff2a4d4f1c6c3e853deab7/src/ring.rs#L212) (edited)

2024-09-24 18:26 darentuzi: Hi, two questions about preimage lookups for service accounts: 1. Is all the raw data, meaning preimage data itself stored in the preimage dictionary, or can it reference external storage? 2. Is the preimage data being deleted when unrequested, and if not, when requested again, is it available within the service or needs to be supplied from an external source?

2024-09-24 20:16 danicuki:

2024-09-24 20:16 danicuki:

2024-09-24 20:17 danicuki:

2024-09-25 03:11 xlchen: I am trying to figure out what is the best way to know if I am the block author of the next block and it seems a bit tricky for the first block on an epoch. γs' determines the block authors and it is only updated at the first block on an epoch. That means for validators to check if they are the block author, they need to figure out γs' first and then check for it.

2024-09-25 03:29 xlchen: What I really want is at begin of an epoch, check γs, and then I know exactly when I need to spawn the block producing task in this epoch. However this won't work for the edge case.

2024-09-25 06:09 gav: 1. Data available for lookup is always stored in the service account state. 2. Whether and when it is deleted from service account state is determined according to the relevant logic described in the host function forget. Once deleted from state it must be resupplied to the chain if a fresh request arrives. (edited)

2024-09-25 07:39 bamzedev:

2024-09-26 17:03 danicuki: We have a question about the JAM prize payment. Rule 20:

Prizes are paid to the earliest Polkadot /
 Kusama account IDs stated in the
 repository's README. In the case of a tie,
 payment is split equally. Local (Swiss) law
 requires that Web3.0 Foundation take
 KYC/AML information on the recipient
 together with proof of account control.

What if the prize will be divided between individuals? Even if these individuals create a multisign wallet, the wallet itself does not belong to any specific entity (individual or company). Who should KYC? All multisign signers? Do individuals need to create an entity that "owns" the multisign wallet? (edited)

2024-09-26 17:09 tomusdrw: > <@danicuki:matrix.org> We have a question about the JAM prize payment. Rule 20: > >

> Prizes are paid to the earliest Polkadot /
>  Kusama account IDs stated in the
>  repository's README. In the case of a tie,
>  payment is split equally. Local (Swiss) law
>  requires that Web3.0 Foundation take
>  KYC/AML information on the recipient
>  together with proof of account control.
> 

> > What if the prize will be divided between individuals? Even if these individuals create a multisign wallet, the wallet itself does not belong to any specific entity (individual or company). Who should KYC? All multisign signers? Do individuals need to create an entity that "owns" the multisign wallet? Curious as well. Note that the tokens are also vested so it needs to be not a one-off distribution between multisig members, but rather a 24-months+ active collective

2024-09-26 22:55 gav: > <@danicuki:matrix.org> We have a question about the JAM prize payment. Rule 20: > >

> Prizes are paid to the earliest Polkadot /
>  Kusama account IDs stated in the
>  repository's README. In the case of a tie,
>  payment is split equally. Local (Swiss) law
>  requires that Web3.0 Foundation take
>  KYC/AML information on the recipient
>  together with proof of account control.
> 

> > What if the prize will be divided between individuals? Even if these individuals create a multisign wallet, the wallet itself does not belong to any specific entity (individual or company). Who should KYC? All multisign signers? Do individuals need to create an entity that "owns" the multisign wallet? More of a question for W3F financial. In principle it is a prize, nothing more. W3F will do what it can to make things smooth but don’t expect anything more than one payment to an account or entity nominated by that person or persons who prove(s) ownership/control/knowledge of codebase.

2024-09-26 22:55 gav: IANAL but I’d expect KYC would be on those persons which the W3F could reasonably know or suspect are in receipt of funds.

2024-09-28 19:09 danicuki: In Formula 139: on part (kv)E, shouldn't this "E" be lowercase as defined in formula 54?

2024-09-28 19:10 danicuki:

2024-09-29 09:19 gav: In fact the other way around. E was meant to be upper case. Feel free to make an issue in the repo. (edited)

2024-09-29 11:28 danicuki: > <@gav:polkadot.io> In fact the other way around. E was meant to be upper case. Feel free to make an issue in the repo. Do you mean e should be uppercased here: https://graypaper.fluffylabs.dev/#WyIzODcxMDNkODIzIiwiZCIsIkJsb2NrIFByb2R1Y3Rpb24gYW5kIENoYWluIEdyb3d0aCIsIktleSBSb3RhdGlvbiIsWyI8ZGl2IGNsYXNzPVwidCBtMCB4ZmUgaGMgeTRlOCBmZjE1IGZzMCBmYzAgc2MwIGxzMCB3czBcIj4iLCI8ZGl2IGNsYXNzPVwidCBtMCB4NzEgaGQgeTRlOSBmZjE3IGZzNSBmYzAgc2MwIGxzMCB3czBcIj4iXV0= ?

2024-09-29 11:38 gav: Yes

2024-10-07 03:10 xlchen: A question about timing. Say current time t = 60, timeslot = 10, for block author of timeslot 11, when should it start producing block? what time is the deadline? say we provide 2s for block production (what is the max time allowed for block production?), then the block author should start producing block at t = 66 and sending blocks at t = 68? other validators should receive block by t = 70 and have it confirmed before t = 72? How about guaranteeing? For guarantors of cores at timeslot 11, what time should it start producing work report and what's the deadline of sending it out? Does it need to make sure the work report is available at t = 66 so the block author can accumulate it in block #11? So it should start the work at t = 60?

2024-10-07 06:57 gav: A block of time slot t = 10 becomes valid once the clock reads 60. On this basis, an optimal block author for t=10 will have sent the block to arrive at all other validator nodes by then. Validators should then have distributed all relevant extrinsic information for this block to arrive in time for the author to include it in a block and then send the necessary block information back to them (we’re working on merklising the extrinsic commitment so that not the whole block needs to be distributed if the receiver has the various extrinsic components). We’re still working on having specific timing guidelines as this will be affected by practical matters ; indeed ideal timing probably won’t be properly fixed until we have the toaster running. (edited)

2024-10-07 06:57 gav: @dave:parity.io might be able to shed a bit more light.

2024-10-07 07:09 xlchen: I see. I will put some placeholder values for now.

2024-10-07 07:52 dave: FWIW, at the moment, our node starts authoring at the start of its timeslot. Guaranteeing is fairly flexible as a block author can include reports from both the current and previous core rotations. I expect guarantor timing will be mostly down to builders as, unless guarantors are already busy with previous packages, there seems little point in waiting to refine a package. As Gav says though I don't think we will have a good idea of when things should be scheduled until we have a test network running, so I would just make sure any values are easy to change.

2024-10-07 08:13 voliva: Hi everyone! I wanted to share that after reading about decentralized JAM, Whitespace caught my curiosity. So, during my spare time, I decided to give it a try and implemented Keccak-256 in Whitespace. The code is available here under GPLv3: https://github.com/voliva/wsa/blob/main/examples/keccak.ws. It's an initial version, and it may have a few bugs, but for the few words I've tried, it hashes them correctly. To facilitate this, I've also built a Whitespace assembler/interpreter/debugger, which helped me work on it. It’s released under the MIT license here: https://github.com/voliva/wsa. Additionally, I've developed a memory layout convention and a "standard library" to make it a bit more practical. I thought I'd share this in case someone finds it interesting. I don't think I'll be building JAM (probably not even the first milestone) in Whitespace, as I really don't have time for it, but I find it fun to solve small objectives in esoteric languages like these.

2024-10-07 10:16 gav: > <@voliva:matrix.org> Hi everyone! > > I wanted to share that after reading about decentralized JAM, Whitespace caught my curiosity. So, during my spare time, I decided to give it a try and implemented Keccak-256 in Whitespace. The code is available here under GPLv3: https://github.com/voliva/wsa/blob/main/examples/keccak.ws. It's an initial version, and it may have a few bugs, but for the few words I've tried, it hashes them correctly. > > To facilitate this, I've also built a Whitespace assembler/interpreter/debugger, which helped me work on it. It’s released under the MIT license here: https://github.com/voliva/wsa. Additionally, I've developed a memory layout convention and a "standard library" to make it a bit more practical. > > I thought I'd share this in case someone finds it interesting. I don't think I'll be building JAM (probably not even the first milestone) in Whitespace, as I really don't have time for it, but I find it fun to solve small objectives in esoteric languages like these. awesome :) what's the next thing to implement?

2024-10-07 19:54 danicuki: Does anybody have the vectorized version of the JAM logo combined with the Polkadot - I found only this one: https://github.com/w3f-webops/graypaper-website/blob/main/src/images/faviconjam.svg ? gav (edited)

2024-10-07 20:05 gav: There’s no vectorized logo - it’s a slightly processed photo of my handwriting on paper.

2024-10-07 20:05 gav: I can dig out the png though.

2024-10-07 20:09 tebdev: I plan to use Keccak-256 and other algo from a Crypto Library. Is that allowed?

2024-10-08 00:01 xlchen: Anyone have started implement some RPC? What will be the best way for us to collaborate it so we all implements a same set of RPC methods?

2024-10-08 00:04 xlchen: There are few key decisions I would like to confirm with everyone before I invest too much time on our RPC implementation : - Which RPC protocol? - JSONRPC - gRPC - Restful - something else? - How high level should the RPC be? - e.g. like Substrate, the RPC provide metadata and raw data and it is up to the SDK to decode the data - we don't need metadata here as the types are all spec'ed in GP - or make it more complicated but dev friendly by returning typed data directly (e.g. JSON)

2024-10-08 00:09 xlchen: And once we have some RPC implemented, I plan to build some simple TUI/webapp (TDB). Something like substrate telemetry

2024-10-08 01:28 tebdev: Sounds good. I am flexible. You have my proxy since you taking the lead.

2024-10-08 05:01 xlchen: I think we will go for JSONRPC + HTTP. i.e. something similar to Substrate for the reasons: - Target users are only JAM devs and network operators. Note: dapp devs, service providers (e.g. CEX), wallets are NOT target users. They should not deal with JAM chain directly. Therefore most of the downsides of the existing Substrate JSONRPC doesn't apply (e.g. requires heavy SDK to decode data returned from JSONRPC). Block explorer / Indexer may have some interests on fetching JAM chain data but it is still only for devs only. No user should worry about JAM chain data. - Most if not all JAM devs will already be familiar with JSONRPC. - JSONRPC with WebSocket supports pushing, which is a very useful feature. RESTful is out of option then. We could do RESTful + WebSocket but then it will be extra work for benefits not super relevant. This RPC does not need to be easy to use. - JSONRPC is not efficient. But under the context of JAM chain, that's something we can deal with. We may want to have some HTTP methods for methods that needs to return large binary payload to avoid the overhead of hex encode / decode the binary data.

2024-10-08 08:35 tomusdrw: I wonder if it's really worth to have every client build it's own RPC (especially based on JSON+HTTP). Have you considered writing a standalone application that would connect to the node via mmap/IPC or some lower-than-http-and-ws-level API and just speak JAM-codec data, possibly just a subset of JAMSNP? I think that would save a lot of time to implement JSON serialisation and could be re-used between client implementaitons.

2024-10-08 08:36 tomusdrw: Initially it would expose JSONRPC over HTTP/WS just for devs, but in future it could evolve into a user-facing RPC

2024-10-08 09:56 xlchen: Substrate RPC are exposing raw data directly and we can do similar thing here so we don’t need to define the JSON shape

2024-10-08 09:57 xlchen: JAM chain are not for users to access so no need to worry about user friendliness. for example, exchanges only integrate with parachains/jam services but never need to deal with JAM chain

2024-10-08 09:59 xlchen: there must be a way to get data out of the node for diagnostic purpose so all the implementations have to implement some sort of it

2024-10-08 10:00 xlchen: we might as well make the interface to be the same so we can reuse tools. that’s one of the reason I think JSONRPC is better compared to to gRPC due to simplicity

2024-10-08 10:06 oliver.tale-yazdi: > <@xlchen:matrix.org> There are few key decisions I would like to confirm with everyone before I invest too much time on our RPC implementation : > > - Which RPC protocol? > - JSONRPC > - gRPC > - Restful > - something else? > - How high level should the RPC be? > - e.g. like Substrate, the RPC provide metadata and raw data and it is up to the SDK to decode the data > - we don't need metadata here as the types are all spec'ed in GP > - or make it more complicated but dev friendly by returning typed data directly (e.g. JSON) JSONRPC + Websocket sounds good to me, i would integrate this into our node if it existed. About metadata: for most of them we probably dont need it, only if we want to have extensions or something later that should be forward-compatible with future clients. I think for the beginning maybe not needed?

2024-10-08 10:08 tomusdrw: > <@xlchen:matrix.org> Substrate RPC are exposing raw data directly and we can do similar thing here so we don’t need to define the JSON shape I get your point. What I'm arguing about is: 1. Why wrap binary data into JSON if we can just return binary data directly. 2. Why introduce another layer of API methods if we have JAMSNP already that should cover the case of getting and even announcing required raw data from the node. 3. Why integrate HTTP/WS server into the node if this can be a middleware that's shared between all implementations. So my idea would be to: 1. define a low-level IPC/mmap socket that can speak JAMSNP (or some subset of it) (every implementation) 2. have a "RPC app" connect to that socket and expose JSONRPC or whatever more human/browser friendly protocol to display the data (shared between implementations)

2024-10-08 10:16 emielsebastiaan: JSON introduces an extra layer of opinionation that you don’t want for testing purposes I suppose. All implementation teams should be capable of working with the codec data.

2024-10-08 10:18 xlchen: I see. JAMSNP is basically jamcodec over QUIC. Then we can probably find some existing proxy convert QUIC to websocket or the node can just doing jamcodec over websocket directly to serve requests to browser (edited)

2024-10-08 10:21 xlchen: the only downside is that to build a dashboard, we need to implement jamcodec in JS to parse the data. but hopefully one of the nodejs implementation team can provide the codec part for use

2024-10-08 10:25 tomusdrw: > <@xlchen:matrix.org> I see. JAMSNP is basically jamcodec over QUIC. Then we can probably find some existing proxy convert QUIC to websocket or the node can just doing jamcodec over websocket directly to serve requests to browser Either this or have the nodes create this IPC/memory-mapped file endpoint that can be used. You don't need QUIC for that, pure in-out, new-line delimited, etc.

2024-10-08 10:26 xlchen: that will require two binary to run side by side, which may cause some trouble on some deployment setup

2024-10-08 10:27 tomusdrw: > <@xlchen:matrix.org> the only downside is that to build a dashboard, we need to implement jamcodec in JS to parse the data. but hopefully one of the nodejs implementation team can provide the codec part for use Ah, that's true. That could either be done in the shared middleware or if there is something that could be used out of the box to just translate the transport then yes, that would be in the app. We do have the codec ready and we happen to write in TypeScript - I'm happy to publish it separately to npm.js It wasn't my intention to plug our solutions stuff here, so sorry about that.

2024-10-08 13:12 tebdev: > <@xlchen:matrix.org> There are few key decisions I would like to confirm with everyone before I invest too much time on our RPC implementation : > > - Which RPC protocol? > - JSONRPC > - gRPC > - Restful > - something else? > - How high level should the RPC be? > - e.g. like Substrate, the RPC provide metadata and raw data and it is up to the SDK to decode the data > - we don't need metadata here as the types are all spec'ed in GP > - or make it more complicated but dev friendly by returning typed data directly (e.g. JSON) Please excuse, I am a little confused. How will this help collaboration and avoid feedback becoming collusion? What is the incentive?

2024-10-08 13:20 oliver.tale-yazdi: > <@tebdev:matrix.org> Please excuse, I am a little confused. How will this help collaboration and avoid feedback becoming collusion? What is the incentive? From what i understood this targets non-GP functionality, like logs monitoring and telemetry etc

2024-10-08 13:33 tebdev: Please excuse, could logs/monitoring/telemetry indicate health and status. If problem detected and team T(i) makes suggestion to T(i+1) could that be collusion? The incentive of "prize reduction to boost network security" may need clarification so we can collaborate without a lawyer ;) If I am reading rules wrong, please correct me.

2024-10-08 13:40 oliver.tale-yazdi: yea it could be good to have some examples of things that teams are allowed to work on together (if any). (edited)

2024-10-08 13:42 tebdev: I am eager to collaborate - Strength in Numbers. The prize is not my primary motivation, but it is an incentive ;)

2024-10-08 16:03 lucas: Any Gossamer team members interested in chatting? I am interested in networking with the Gossamer team.

2024-10-08 17:03 gav: Anything not specified nor referenced in the GP is probably fine. If you don’t need to do it for a milestone then it’s fine to chat about. (edited)

2024-10-08 17:06 gav: Examples: database structure and formats, CLI arguments, RPCs, xUIs.

2024-10-08 20:56 tebdev: > <@gav:polkadot.io> Examples: database structure and formats, CLI arguments, RPCs, xUIs. Sounds reasonable. Focus on interface agreements and treat GP as black box. There is value in that.

2024-10-11 12:58 sebastian: > <@danicuki:matrix.org> We have a question about the JAM prize payment. Rule 20: > >

> Prizes are paid to the earliest Polkadot /
>  Kusama account IDs stated in the
>  repository's README. In the case of a tie,
>  payment is split equally. Local (Swiss) law
>  requires that Web3.0 Foundation take
>  KYC/AML information on the recipient
>  together with proof of account control.
> 

> > What if the prize will be divided between individuals? Even if these individuals create a multisign wallet, the wallet itself does not belong to any specific entity (individual or company). Who should KYC? All multisign signers? Do individuals need to create an entity that "owns" the multisign wallet? Hi danicuki. For payments, see rule 8.5 of the T&Cs. As for KYC, all beneficiaries have to KYC if no legal entity exists.

2024-10-11 14:23 dave: I've uploaded a more fleshed out definition of the "simple" network protocol for JAM here: https://github.com/zdave-parity/jam-np/blob/main/simple.md

2024-10-11 14:26 dave: I do not intend to change this any more, except to correct errors, and to add stream protocols for Grandpa (which is not currently specified). Any changes for e.g. efficiency will be made in the full network protocol, which won't be defined any time soon

Vec<Hash> ++ Blob ++ Vec<Hash> ++ Vec<SegmentChunk> ++ Vec<Hash>

The Vec<Hash> will just be complementary Merkle-node-hashes from leaf to root.  The first will contain hashes for the blob-subtree, the second for the segments subtree and the third for the super-tree.

> Vec<Hash> ++ Blob ++ Vec<Hash> ++ Vec<SegmentChunk> ++ Vec<Hash>
> 
> The Vec<Hash> will just be complementary Merkle-node-hashes from leaf to root.  The first will contain hashes for the blob-subtree, the second for the segments subtree and the third for the super-tree.
> 

2024-10-11 14:26 dave: AFAIK Gav does not intend to include this initial "simple" network protocol in the Gray Paper; only the full network protocol will be included, once it has been defined

2024-10-11 14:27 dave: So as far as the simple protocol is concerned, the linked repo can be considered definitive

2024-10-11 15:25 oliver.tale-yazdi: > <@dave:parity.io> I've uploaded a more fleshed out definition of the "simple" network protocol for JAM here: https://github.com/zdave-parity/jam-np/blob/main/simple.md Included it here: https://docs.jamcha.in/advanced/networking/SNP I will try to keep aggregating all non-graypaper docs in there for easy discovery in the future

2024-10-11 22:55 danicuki: Shouldn't this test vector have an error code? infinite: 1, halt: 2, bad: 3, big: 4 https://github.com/w3f/jamtestvectors/blob/master/codec/data/work_result_1.json (panic: null) should be (panic: <number>) or am I missing something?

2024-10-11 23:01 rick: Generally speaking they are "syntactically correct only" according to the README, I took that to mean per the binary format it decodes.. In this case, WorkExecResult in the ASN defines panic as null and not having a byte sequence: https://github.com/w3f/jamtestvectors/blob/master/codec/schema.asn#L93 The only one that has a specified by sequence is the "ok" result. So "panic": null looks correct to me as none of the errors have byte sequences. (edited)

2024-10-11 23:10 danicuki: understood. A suggestion for changing this would be WorkExecResult ::= CHOICE { ok \[0\] ByteSequence, error \[1\] INTEGER } and return the error number: out\_of\_gas: 1, panic: 2, bad-code: 3, code-oversize: 4 because the way it seems ambiguous. E.g. when bad-code is not defined, it is null, as well as the panic being explicitly defined as null (edited)

2024-10-11 23:20 rick: > <@danicuki:matrix.org> understood. A suggestion for changing this would be > > WorkExecResult ::= CHOICE { > ok \[0\] ByteSequence, > error \[1\] INTEGER > } > > and return the error number: > out\_of\_gas: 1, panic: 2, bad-code: 3, code-oversize: 4 > > because the way it seems ambiguous. E.g. when bad-code is not defined, it is null, as well as the panic being explicitly defined as null When it's an error, the binary encodes 0x01, 0x02, 0x03, or 0x04. When it's not, it encodes 0x00 [len encoding] [bytesequence]. This is proposing 0x01 [4 byte integer], so an additional 4 bytes when there is an error. That doesn't seem unreasonable to me, but worth mentioning.

2024-10-12 08:41 danicuki: > <@rick:carback.us> When it's an error, the binary encodes 0x01, 0x02, 0x03, or 0x04. When it's not, it encodes 0x00 [len encoding] [bytesequence]. This is proposing 0x01 [4 byte integer], so an additional 4 bytes when there is an error. That doesn't seem unreasonable to me, but worth mentioning. Understood. The binary indeed is better to keep small as you mentioned. I was just wondering if there is another way of the json be more intuitive.

2024-10-12 11:10 danicuki: rick do you know whether the work_item and work_package test vectors already updated to the latest GP version (0.4.1)?

2024-10-12 14:05 rick: > <@danicuki:matrix.org> rick do you know whether the work_item and work_package test vectors already updated to the latest GP version (0.4.1)? No, sadly I am just a lowly implementer like yourself 😅

2024-10-15 14:45 luke_f: Hello. is chapter 19, Grandpa and the Best chain part of the requirements for milestone 1?

2024-10-16 09:03 gav: No.

2024-10-16 09:05 gav: > <@danicuki:matrix.org> rick do you know whether the work_item and work_package test vectors already updated to the latest GP version (0.4.1)? I expect they’re still 0.3 since we have not yet finished the changes for 0.4 internally yet.

2024-10-16 09:07 gav: Fwiw, I’ll be bringing a few limited-edition 0.4.2 “Lisbon” gray paper prints to the Monday night Lisbon Jam meetup.

2024-10-16 13:24 danicuki: For those JAM implementors who will be at the meetup next Monday, would be nice to have you for 10min on stage sharing your experience with the audience. Please reach out to me ASAP so I can organize the agenda.

2024-10-16 13:28 danicuki: From what I saw in the event subscription, we will have people from these teams: PyJAMaz, JAM Zig, GrayMatter, Gossamer and Jamixir. Please reach out

2024-10-17 16:32 dave: > <@dave:parity.io> I've uploaded a more fleshed out definition of the "simple" network protocol for JAM here: https://github.com/zdave-parity/jam-np/blob/main/simple.md FYI I've made a fix to the epoch transition section and tweaked relevant bits of the Safrole ticket protocol section. Diff here: https://github.com/zdave-parity/jam-np/commit/6d2cba57f45fb1af52baab6618c9a32f71dc185b

2024-10-17 19:56 stanleyli: Hi David Emett (also davxy), sorry I still have questions about work package and work package bundle: (1) For the import_segments sitting inside of work_item. what's the definition of tree_root & index in import_segments? Is the tree_root here referring to certain work report's exported_segment_root and exported_segment_idx, which will include justification going up erasure_root(u), or is this are the exported_segment_root with justification going up to expoerted_segment_root(e)? https://github.com/w3f/jamtestvectors/blob/master/codec/data/work_package.json#L22-L62 { "tree_root": "0x461236a7eb29dcffc1dd282ce1de0e0ed691fc80e91e02276fe8f778f088a1b8", "index": 0 } (2) When looking at published codec data, I can see both work package have workItem containing importedSegment always starting from index 0 ... up to 3 with no gap. Is it a coincidence, or index here actually mean the index within a workItem? (3) In CE134, you mentioned "All import segments have been retrieved..." and "a full work-package bundle is sent, along with any necessary work-package hash to segment root mappings". [Work Package Hash ++ Segment Root] how does this allow other guarantor to validate anything? Also what does the original Guarantor have to do to get the exportedSegment, by doing CE 139? (4) Is there a work_package_bundle data available so we can make sure we got the format correct conforming to 14.1.1 in GP 0.4.2 and your jam_np?

2024-10-18 07:08 xlchen: I am confused about JAMNP ALPN > The protocol name, version, and chain are identified using QUIC/TLS "ALPN" (Application Layer Protocol Negotiation). The protocol identifier should be either jamnp-s/V/H or jamnp-s/V/H/builder. Here V is the protocol version, 0, and H is the first 8 nibbles of the hash of the chain's genesis block header. firstly, I guess the V and H are ASCII encoded? so it will be jamnp-s/0/12345678 encoded using ASCII?

2024-10-18 07:09 xlchen: And shouldn't it be using genesis state hash rather genesis block header hash? surely it can't be the first block in the network

2024-10-18 07:13 xlchen: so just searched genesis header in GP so I guess it will be something predefined along with genesis state. i.e. something should be included the chainspec file

2024-10-18 07:51 dave: > <@xlchen:matrix.org> I am confused about JAMNP ALPN > > The protocol name, version, and chain are identified using QUIC/TLS "ALPN" (Application Layer Protocol Negotiation). The protocol identifier should be either jamnp-s/V/H or jamnp-s/V/H/builder. Here V is the protocol version, 0, and H is the first 8 nibbles of the hash of the chain's genesis block header. > > firstly, I guess the V and H are ASCII encoded? so it will be jamnp-s/0/12345678 encoded using ASCII? Yes

2024-10-18 07:55 dave: > <@xlchen:matrix.org> so just searched genesis header in GP so I guess it will be something predefined along with genesis state. i.e. something should be included the chainspec file Yes

2024-10-18 11:33 sourabhniyogi: David Emett: We need a way for users interact with “always accumulate” services, at least the assign privileged service right now but maybe all of them in the same way. Should we use CE130 to prototype this on our own or maybe you have a design in mind?

2024-10-18 11:35 dave: > <@sourabhniyogi:matrix.org> David Emett: We need a way for users interact with “always accumulate” services, at least the assign privileged service right now but maybe all of them in the same way. Should we use CE130 to prototype this on our own or maybe you have a design in mind? I don't understand what you mean by this

2024-10-18 11:35 gav: sourabhniyogi: please read the Code of Conduct effective across all public Polkadot channels (edited)

2024-10-18 11:36 gav: > This is a room for informal and inclusive conversation. It is not an opportunity to call out for a **specific individual** to answer arbitrary questions. Questions that are made directly to a room member not already in direct conversation with the asker will not be tolerated. (edited)

2024-10-18 11:37 gav: If you have a question, ask it. Do not address a particular person unless as part of a pre-existing and on-topic conversation.

2024-10-18 11:47 sourabhniyogi: > <@gav:polkadot.io> If you have a question, ask it. Do not address a particular person unless as part of a pre-existing and on-topic conversation. Sorry I was asking as a follow up to a preexisting conversation from a week ago. How are users supposed to interact with the assign service or privileged/ always accumulate services?

2024-10-18 11:51 gav: In the same way as any other service. (edited)

2024-10-18 11:51 gav: By sending Work Packages.

2024-10-18 11:52 gav: There will need to be a "bootstrap" service, specified in the genesis config and already in state, which accepts simple Work Packages with instructions in them, such as for assigning coretime, creating new services, requesting preimages and changing the validator keysets. (edited)

2024-10-18 11:53 gav: We already have an early revision of this internally (just used for tests at present) and will tidy up and publish in due course. (edited)

2024-10-18 11:54 gav: It doesn't make much sense to have this service sitting around until we have a testnet going.

2024-10-18 12:27 sourabhniyogi: We have single-team testnet and can definitely have multi-team testnets after getting this genesis state containing the bootstrap service. Definitely ready. We have our "tiny" V=6/C=2 testnet with CE133/134/135 (work package submission/sharing => work report distribution) with 1 core being assured now via CE141 and hitting accumulate entry points, updated with the new JAMSNP details. We currently have "hard coded" a fib test service into state trie (via the service's preimage) -- we can totally improve this to have a genesis state with this "bootstrap" assign service (with a new host call) instead of our current "hard coded" situation. Ok -- so my questions concerning privileged / always accumulate services (which are not GP but JAMSNP related, but its blurry here): 1. if a user sends their Work Package into the bootstrap assign service via CE133 containing their service code blob, presumably to any validator on any core (since the assign service does not belong to a core ... or does it?), what happens next? 2. In JAMSNP, CE134 has the first guarantor sharing with 2 others, resulting in a work report distribution -- should it share with all V-1 instead if its known to be among the 3 privileged or always-accumulate services in \chi_g? 3. Does a work report need to be distributed at all then via CE 135 for the privileged / always accumulate service? 4. Can CE 141 be skipped then? We will take the answers to this and accommodate in our revisiting "Accumulate" (to cover ordered accumulation since we finally have real assurances in our testnet), covering both the normal work packages and the above "always-accumulate". We can publish our genesis state and believe many other teams can use the same genesis state + publish exact same answers to get single-team testnets and then proceed to multi-team testnets now. (edited)

2024-10-18 12:39 dave: AFAIK there is nothing special about work packages containing items for always-accumulate services. They are guaranteed and audited in the same way as any other work package

2024-10-18 13:07 gav: There is indeed nothing special about privileged services. Also note that privileged services do not always accumulate. There is now (GP-0.4) a separate item which contains a mapping of always-accumulate services and the free gas they get. (edited)

2024-10-18 13:09 gav: 1. There will likely be a single bootstrap service which is given all three privileges.

2024-10-18 13:11 gav: There’s nothing special about the service (other than being privileged). The WP will be processed as long as there is a core whose assignment is acceptable. Probably we’ll start with a null-authorizer assigned to all cores which allows any core to accept any WP. (edited)

2024-10-18 13:12 gav: 2. It shouldn’t do anything different.

2024-10-18 13:12 gav: 3. Same.

2024-10-18 13:12 gav: 4. Same.

2024-10-18 13:14 davxy: > <@stanleyli:matrix.org> Hi David Emett (also davxy), sorry I still have questions about work package and work package bundle: > > (1) For the import_segments sitting inside of work_item. what's the definition of tree_root & index in import_segments? Is the tree_root here referring to certain work report's exported_segment_root and exported_segment_idx, which will include justification going up erasure_root(u), or is this are the exported_segment_root with justification going up to expoerted_segment_root(e)? > > https://github.com/w3f/jamtestvectors/blob/master/codec/data/work_package.json#L22-L62 > { > "tree_root": "0x461236a7eb29dcffc1dd282ce1de0e0ed691fc80e91e02276fe8f778f088a1b8", > "index": 0 > } > > (2) When looking at published codec data, I can see both work package have workItem containing importedSegment always starting from index 0 ... up to 3 with no gap. Is it a coincidence, or index here actually mean the index within a workItem? > > (3) In CE134, you mentioned "All import segments have been retrieved..." and "a full work-package bundle is sent, along with any necessary work-package hash to segment root mappings". [Work Package Hash ++ Segment Root] how does this allow other guarantor to validate anything? Also what does the original Guarantor have to do to get the exportedSegment, by doing CE 139? > > (4) Is there a work_package_bundle data available so we can make sure we got the format correct conforming to 14.1.1 in GP 0.4.2 and your jam_np? The content of the **codec** test vectors is not supposed to be significant. These vectors are only used to test your codec encode/decode functions https://github.com/w3f/jamtestvectors/tree/master/codec#semantic-correctness

2024-10-18 13:14 gav: The above should all be clear from reading the GP carefully.

2024-10-18 13:14 sourabhniyogi: Alright, thank you for clarifying, this is simpler. Is the bootstrap service assigned to just one core?

2024-10-18 13:14 gav: Cores are not assigned to services (edited)

2024-10-18 13:14 gav: Nowhere in the GP is this implied.

2024-10-18 13:15 gav: Cores are assigned to authorizers. Authorizers authorize (sets of) work packages. Work packages contain work items. Work items are associated with a service. (edited)

2024-10-18 13:15 sourabhniyogi: Ok understood, thank you.

2024-10-18 13:49 carlos379:

2024-10-18 15:31 dvladco: Hello, I have a question regarding the milestone 1. What does "PVM instancing" imply? having a general PVM that will be later used for the four types of invocations (is-authorized, refine, accumulate, on-transfer) or does this mean implementing each of the four invocation types. In short does "PVM instancing" mean implementing only Ψ and ΨH and perhaps ΨM or does that also involve ΨI, ΨA, ΨR, ΨT? (edited)

2024-10-18 15:58 gav: The former.

2024-10-18 15:59 gav: But it must be possible to implement each of the other things with it.

2024-10-18 16:00 gav: (M1 necessarily includes the last two PVM invocation functions too since they’re used on-chain)

2024-10-18 16:01 dvladco: last two meaning accumulate and on-transfer?

2024-10-18 16:01 gav: Yes

2024-10-18 16:01 dvladco: thanks

2024-10-18 16:04 dvladco: I was trying to implement the is-authorized first but it required work packages, cores and what not, so I was not sure if we needed all that for the M1, thanks for clarifying it 🙂

2024-10-18 20:16 xlchen: about JAMNP, can we have explicit length prefix for CE stream request/response and length prefix for UP messages?

2024-10-18 20:18 xlchen: because otherwise it will be much more work to be able to reject bad data

2024-10-18 20:19 xlchen: many streams have implicit fixed size data so they maybe don’t need such prefix, but in that case will be good to explicit note the expected message size in the spec

2024-10-18 21:59 dave: Possibly I misunderstand, but all messages are length prefixed on the wire: "A message is transmitted in two parts. First, the size (in bytes) of the message content is transmitted, encoded as a little-endian 32-bit unsigned integer. Second, the message content itself is transmitted."

2024-10-19 04:53 xlchen: my bad. not sure why I missed that part 🤦‍♂️

2024-10-19 15:16 gav:

2024-10-19 15:56 gav: Services are not smart contracts. JAM is not intended to be used directly as an application chain since it’s primarily a roll-up host. If you’re desperate to get information out of JAM you can write to a service’s storage and inspect that.

2024-10-19 18:59 sourabhniyogi: Understood about JAM services not intended to be used directly as application chain. But what about the bootstrap service itself: how should a new service creator learn that the newly created service has a particular new service index (from check/bump entropy driven processes, via 4.2 (281)'s check ala hostfunction new after accumulate completes). Do you mean that a new creator of a service should look in the bootstrap service's storage? This is fine for PoC -- but do you mean for production multifunction bootstrap service as well? (edited)

2024-10-21 11:07 gav: > <@sourabhniyogi:matrix.org> Understood about JAM services not intended to be used directly as application chain. > > But what about the bootstrap service itself: how should a new service creator learn that the newly created service has a particular new service index (from check/bump entropy driven processes, via 4.2 (281)'s check ala hostfunction new after accumulate completes). > > Do you mean that a new creator of a service should look in the bootstrap service's storage? This is fine for PoC -- but do you mean for production multifunction bootstrap service as well? Production would be different. Most likely it will be the parachains service which is given all three privileges. The Staking/CoreTime chains would be able to set validators, assign cores, request blobs and create services.

2024-10-21 11:08 gav: > how should a new service creator learn that the newly created service has a particular new service index The most obvious way (and what I'm doing in my little test) is to write to a known storage key the new index.

2024-10-21 11:23 gav: FWIW logs/receipts are basically an unsolved problem in blockchain. It's not at all clear how to build a secure decentralised scalable service capable of indexing arbitrarily large numbers of state-change events. Ethereum's bloom filter became saturated shortly into its lifetime. Deterministic hash-maps are trivially spammable. Non-deterministic hash-maps are difficult to validate. Maybe there's something clever which can be done with ZK these days, but overall it's a hard one to solve and not one I especially want to bundle into the core of JAM.

2024-10-22 09:18 prematurata: Not sure if this is the right place and if it is allowed to ask. But I was thinking it could be helpful and cool to have some "voting" support for jam teams right now in the polkadot chain as a perk for jam implementors. Especially now that the number of validators in the dot chain was (and is being) raised.

2024-10-22 10:31 gav: > <@prematurata:matrix.org> Not sure if this is the right place and if it is allowed to ask. But I was thinking it could be helpful and cool to have some "voting" support for jam teams right now in the polkadot chain as a perk for jam implementors. > > Especially now that the number of validators in the dot chain was (and is being) raised. To some extent this is already provided for via the Fellowship membership rules. But I suppose there could be an additional collective for the maintainers of all milestone-achieving teams.

2024-10-22 12:14 jaymansfield: Hey all. Should we be updating anyone at w3f about our progress as we go or only after completing milestones?

2024-10-22 12:58 gav: Feel free to give updates - we’re interested to know.

2024-10-22 15:24 dakkk: > <@jaymansfield:matrix.org> Hey all. Should we be updating anyone at w3f about our progress as we go or only after completing milestones? It would be interesting to know at what stage are all the teams working on a jam implementation

2024-10-22 15:24 dakkk: (at least those using the language I'm using 😄)

2024-10-22 17:22 gav: JAM should be easily performant enough to be able to verify most Ethereum L2 transitions in a single WP. Along with a secure ethereum bridge, it should be viable to provide cheap, fast and secure Ethereum L2s from JAM. It may even be possible to grandfather in pre-existing L2s.

2024-10-22 17:24 gav: It’s still in the early stages of ideation but I don’t see any obvious flaws with the approach. It is perhaps something which could be discussed at JAM0.

2024-10-22 22:54 danicuki: ima_8984eb0.jpeg

2024-10-22 22:54 danicuki: ima_779bc41.jpeg

2024-10-22 22:54 danicuki: ima_ab4d75d.jpeg

2024-10-22 22:54 danicuki: ima_53184c7.jpeg

2024-10-22 22:54 danicuki: ima_ddab933.jpeg

2024-10-22 22:54 danicuki: ima_e241c29.jpeg

2024-10-22 22:54 danicuki: ima_43b6c06.jpeg

2024-10-22 22:54 danicuki: ima_b13309a.jpeg

2024-10-22 22:54 danicuki: ima_1f92268.jpeg

2024-10-22 22:54 danicuki: ima_03602d0.jpeg

2024-10-22 22:54 danicuki: video_af05d68.mp4

2024-10-22 22:54 danicuki: Pictures from the JAM Meetup yesterday in Lisbon.

2024-10-23 04:24 lucas: I am intersted in chatting with Gossemer team members. I am also interested in networking with other JAM engineers. I saw that a solo JAM implementation may also be desired, & beneficial. I am also wondering if there are any set vcs practices that aren't mentioned anywhere on the relevant web resources? Is there any universal agreement among teams with VCS or other engineering practices? What happens if I am engineering solo, parts of my implementation are copied by another team & put in their implementation? How would anyone know if it was originally my implementation vs theirs or vice versa? Wondering as well if anyone is working on JAM related projects that aren't specifically implementations? I am also wondering if I build for example a dApp or something similar on top of a new JAM client implementation if that is more desired than for example an improved JAM client implementation by itself even if the client may perform better by focusing only on that & not building on top of it? For example, I think it'd be cool to build a decentralised gaming server, that could also be a good experiment to test decentralised network performance & benchmark it against non decentralised server clusters. Some questions that can be explored for example, are all server clusters a form of decentralisation if some resources are distributed or only if they are distributed in a certain way? What defines a decentralised http or protocol request? I haven't discovered any universal answers for the above, or agreement related to them. Is there a stack exchange or similar forum, where it is better to ask these questions?

2024-10-23 04:28 xlchen: I didn't read all the fine print of the prize rule but I assume copyright infringement is an immediate disqualify. KYC/KYB is required so it should be possible to take legal action if necessary

2024-10-23 04:29 xlchen: we definitely need all sorts of tools and applications built on top of JAM but I will say it is a bit too early to actually working on it until we have a public JAM testnet

2024-10-23 04:37 xlchen: JAM is the foundation. it is very low level. users are not suppose to use JAM directly. we need all sorts of services to make JAM useful. however, to build such service, we also need some framework for it. I presume Parity is working on some internal experimental framework and it will take time and many trial and error for such framework to be mature enough for use

2024-10-23 06:07 bill: > <@lucas:digitale-gesellschaft.ch> I am intersted in chatting with Gossemer team members. > > I am also interested in networking with other JAM engineers. > > I saw that a solo JAM implementation may also be desired, & beneficial. > > I am also wondering if there are any set vcs practices that aren't mentioned anywhere on the relevant web resources? Is there any universal agreement among teams with VCS or other engineering practices? What happens if I am engineering solo, parts of my implementation are copied by another team & put in their implementation? How would anyone know if it was originally my implementation vs theirs or vice versa? > > Wondering as well if anyone is working on JAM related projects that aren't specifically implementations? > > I am also wondering if I build for example a dApp or something similar on top of a new JAM client implementation if that is more desired than for example an improved JAM client implementation by itself even if the client may perform better by focusing only on that & not building on top of it? > > For example, I think it'd be cool to build a decentralised gaming server, that could also be a good experiment to test decentralised network performance & benchmark it against non decentralised server clusters. > > Some questions that can be explored for example, are all server clusters a form of decentralisation if some resources are distributed or only if they are distributed in a certain way? > > What defines a decentralised http or protocol request? > > I haven't discovered any universal answers for the above, or agreement related to them. > > Is there a stack exchange or similar forum, where it is better to ask these questions? 1. Yes, there is a 10 million DOT pot available for implementations of JAM. Many different teams are already working on it https://jam.web3.foundation/ 2. As Bryan said, teams that copy/paste from others are disqualified. You can timestamp your code in a variety of ways, e.g., making a system.remark of the latest commit hash at regular intervals. I recommend reading through the rules here: https://jam.web3.foundation/rules

2024-10-23 11:00 emielsebastiaan: Could anyone please update me on JAM-related Bangkok events in November? The bits and pieces I have: - Last day at sub0 (11 Nov) will focus on JAM - JAM0 scheduled on 12-16 Nov. But treasury proposal got rejected. Is this still happening? If yes is there a rough idea of objectives? Where can I find the latest info and keep updated? 🙏🏻

2024-10-23 13:22 gav: I had understood it was to be on the 9th and 10th November.

2024-10-23 13:49 sourabhniyogi: sub0 is 9th through the 11th -- You should register for a free ticket https://sub0.gg/ and look for JAM related content on the schedule, which is under prep there. 127 Na Ranong Rd. , Khlong Toei, Bangkok is the address. rektormoon can speak to other details not on the site but... Hackerspace is open to all implementers 24h! JAM0 is a mostly open ended hackathon type arrangement with 2-3x/day "sessions" where we cover whatever topics anyone wants for 30-90 mins (e.g at 10:30am and 2pm and 5pm) as a group and any team can network with any other team \[always following official/unofficial rules\]. We have a big room that can hold everyone for up to 80 people on the 11th,12th,15th,16th so we can do whatever we want. On the 13th and 14th we have a smaller room that can hold up to 30. For 11th-16th both rooms are available 9am-9pm. I'm collecting a headcount here so we can buy everyone a lunch or dinner if there is quorum of people: https://docs.google.com/spreadsheets/d/1\_Ar0CWH8cDq\_mAoVkqZ20fXjfNQQ9ziv1jsVJBAfd1c/edit?gid=0#gid=0 You can add your topics \[I put 2\] but no prep is required. If you are a JAM implementer (and have ever asked a JAM-related GP question to qualify you), I'll put you in a JAM0 room so you can find everyone else. We are happy to sponsor the venue + food but have no travel budget. If you have any questions/wishes, you can DM me @sourabhniyogi and we'll do our best to be a good host. Address is 95,370 Sukhumvit Soi 24, Khwaeng Klongton, Khet Klongtoey Devcon 7 is Nov 12th-15th and a 25 min walk from both places. Tickets are sold out but there is a waitlist. I'm not seeing a schedule but everyone is welcome to go back and forth between the JAM0 rooms at Valia and Devcon 7 all through 4 days from 9am-9pm. (edited)

2024-10-23 14:29 dave: > <@dave:parity.io> I've uploaded a more fleshed out definition of the "simple" network protocol for JAM here: https://github.com/zdave-parity/jam-np/blob/main/simple.md FYI I've pushed a fix to CE 142. Previously, only the hash of the preimage was sent, which isn't really enough to check whether the preimage has been requested on chain or not. Now the ID of the requesting service and the length of the preimage are also sent.

2024-10-23 18:20 sourabhniyogi: Excellent, thank you for this fix -- I think its workable to complete the picture and implement a bootstrap service like this now: https://github.com/jam-duna/jamtestnet/blob/main/services/jam-duna/bootstrap/README.md Let me know what you think I got wrong? Or if you have a bootstrap assembly that does it better/right, maybe that is ready for sharing? I understand there are problems that are useful as exercises for the reader, and this is a great one, but perhaps achieving consensus on this is worth it. (edited)

2024-10-24 00:34 charliewinston14: I have a question about "publish-tickets-no-mark-4" which is "Fail: Submit tickets in bad order.". Where in the grey paper can i find information about how to properly order tickets? I see "in order of their implied identifier." on page 15 but unsure what that means exactly.

2024-10-24 00:35 xlchen: they need to be ordered by output in ascending order. it is specified somewhere

2024-10-24 01:31 charliewinston14: > <@xlchen:matrix.org> they need to be ordered by output in ascending order. it is specified somewhere Thanks!

2024-10-25 01:43 xlchen: For JAMNP CE128 block request, does the response have to give exact amount of block requested?

2024-10-25 01:44 xlchen: the response appears to not having a length prefix

2024-10-25 01:44 xlchen: but the wording is that the number is request is maximum so response can give something less than that

2024-10-25 09:55 dave: Yeah the intent is that some number of blocks up to the given maximum is returned

2024-10-25 09:56 xlchen: in that case shouldn’t it have a length prefix?

2024-10-25 09:57 xlchen: so I can decode it as Array<Block> directly with jam codec

2024-10-25 10:13 dave: I was assuming you would decode one block at a time in a streaming fashion, stopping once you had read the entire message. Perhaps an additional length prefix would be useful, that's not particularly clear to me though

2024-10-25 10:20 xlchen: it is lot more complicated to implement an online decoder and for what benefit? save 1 to 2 bytes?

2024-10-25 10:25 xlchen: > len++ preceding a sequence indicates that the sequence should be explicitly prefixed by its length. If a sequence is not preceded by len++, the length is either fixed or implied by context. and in this case, neither the length is fixed or implied by context

2024-10-25 10:33 dave: It is implied by the message length? If you want to read the entire message and decode it into an array I don't see what the difficulty is with this?

2024-10-25 10:34 xlchen: it is doable but the wording is confusing. there is no way to derive the block count purely from total message length

2024-10-25 10:36 dave: > <@xlchen:matrix.org> it is doable but the wording is confusing. there is no way to derive the block count purely from total message length I agree, does that matter?

2024-10-25 10:36 xlchen: it is ok to keep as it is, but you could say something like when the array does not have length prefix, people should repeatedly decode the type until all the data are read (edited)

2024-10-25 10:37 xlchen: I am suggesting to improve the wording to make it less ambiguous otherwise you may get similar questions in future

2024-10-25 10:58 dave: I'll try to clarify this

2024-10-25 13:33 dave: I've added a few sentences covering this particular case, hopefully clearer now

2024-10-27 19:42 danicuki: Screenshot 2024-10-27 at 19.42.22.png

2024-10-27 19:42 danicuki: People pay attention when there is a table comparing different products. We all know JAM is the amalgamation of different breakthrough technologies. I tried to put this in a map, so we can use it when we explain JAM to others. It came from a conversation with ChatGPT, helping me to identify other chains that partially implement what we are doing in full on JAM. Here is the conversation link: https://chatgpt.com/share/671e96b7-1e8c-800f-989a-04acc01f1bee and this is the generated table (food for thought and debate) Feedback is very welcome (edited)

2024-10-27 19:44 danicuki: Blockchain_Technology_Comparison_with_Finalized_Corrections.csv

2024-10-28 09:25 gav: The less ostentatious way would be to implement an Ethereum light client within a JAM service (or just use eg Snowfork, which essentially does that but in a parachain)

2024-10-28 09:48 tebdev: > <@gav:polkadot.io> The less ostentatious way would be to implement an Ethereum light client within a JAM service (or just use eg Snowfork, which essentially does that but in a parachain) Very interesting, Help: would it be safe to say this is probabilistic trust versus absolute as a performance tradeoff?

2024-10-28 10:04 gav: Light clients generally are, yes.

2024-10-28 10:10 gav: The good ones will only rely on sensible assumptions about the behaviour of validators. Most are not so good. A full node would manually verify the state transition and therefore give a solid guarantee of correctness. But regarding finality, it’s not so much about correctness and more about the validators committing to one of (potentially) several options. Such a commitment is generally correct per se and therefore about as secure on light (finality-tracking) as full (block-executing) nodes.

2024-10-28 10:37 tebdev: > <@gav:polkadot.io> The good ones will only rely on sensible assumptions about the behaviour of validators. Most are not so good. A full node would manually verify the state transition and therefore give a solid guarantee of correctness. But regarding finality, it’s not so much about correctness and more about the validators committing to one of (potentially) several options. Such a commitment is generally correct per se and therefore about as secure on light (finality-tracking) as full (block-executing) nodes. Yes non-sensible versus "sensible" in today's world may be more when versus if. ;) Games relying on rational actors is a concern. Bravo on using/funding multi programming languages. That is Nobel prize level stuff towards unstoppable ;) For direct multi-objective incentive mechanisms to counter non-rational actors, could the JAM framework consider incorporating game-theoretic approaches that weigh reputational and non-financial factors? In my humble view and experience NASH is not absolute with state actors when stoppable is a goal..

2024-10-28 11:33 tebdev: image.png

2024-10-28 11:37 tebdev: I could be way off, but I could not find this in Jam paper. Normal payoff with deviation added for irrational ... Mixed strategy. Resilience ... (edited)

2024-10-28 13:00 tebdev: Off-Equilibrium Paths thinking out loud... , Maybe Cantelli's inequality can help bound Dj irrational cost which may need treatment in ELVES. ELVES may deal with this, but not clear. Maybe update Game theory payoff model first as foundation to PoS, then show how ELVES would deal with system sensitivity to irrational actions. Maybe mixed incentives plus redistribution of stake. Slashing seems one dimensional, maybe slash and redistribute. (edited)

2024-10-28 17:21 charliewinston14: Hello. Question about erasure coding in JAM. I’ve broken the blob into pieces, and then those pieces into the octet pairs. The pairs were then converted to 16 bits. I’m now trying to figure out the field element formula. It looks to be the summation of each bit multiplied by “vj” and this is where I’m not sure what value to use. If j = 7, then vj = α14 +α4 +α. What is α here?


2024-10-28 17:21 charliewinston14: Screen Shot 2024-10-28 at 1.20.33 PM.png

2024-10-28 22:30 danicuki: Presentations from teams at the JAM Meetup in Lisbon: https://www.youtube.com/watch?v=7N2x6ulVA-E https://www.youtube.com/watch?v=j-Qf25zuQ8w Unfortunately, the camera didn't catch the two last presentations from PolkaJAM and Jamixir, so we will have to wait for sub0 / jam0 meeting in Bangkok for those.

2024-10-29 04:39 xlchen: https://github.com/open-web3-stack/jam-dashboard we created a simple telemetry dashboard displaying chain head for Boka nodes. I can spend a bit time to document the RPC if anyone else is interested

2024-10-29 04:39 xlchen: Screenshot 2024-10-29 at 5.39.36 PM.png

2024-10-29 07:00 dakkk: > <@xlchen:matrix.org> https://github.com/open-web3-stack/jam-dashboard we created a simple telemetry dashboard displaying chain head for Boka nodes. I can spend a bit time to document the RPC if anyone else is interested I'm interested in the RPC, and would be awesome to have the genesis state you used

2024-10-29 08:21 xlchen: we currently only have a helper to create genesis state in code directly. i can export it in some json format

2024-10-29 17:50 dave: > <@dave:parity.io> I've uploaded a more fleshed out definition of the "simple" network protocol for JAM here: https://github.com/zdave-parity/jam-np/blob/main/simple.md FYI I've made a minor modification to the availability protocols to discriminate elements of the justification sequences. This should hopefully allow simpler implementations. I've also removed some statements which I now realise were not accurate! See https://github.com/zdave-parity/jam-np/commit/7637e963cfd22fc452a957904b2f8d5ff181e67b and https://github.com/zdave-parity/jam-np/commit/032a7f14264c0e441c8e50d9c3c8504d5ac0c62c

2024-10-29 18:01 sourabhniyogi: image.png

2024-10-29 18:02 sourabhniyogi: Is the above suggestion to interpret EVM byte code using PolkaVM sane? I didn't think it was sane until this week with the suggestion that JAM run an Ethereum full node and the above, which is truly ostentatious! This is from https://vitalik.eth.limo/general/2024/10/26/futures5.html (edited)

2024-10-30 09:57 gav: > <@sourabhniyogi:matrix.org> Is the above suggestion to interpret EVM byte code using PolkaVM sane? I didn't think it was sane until this week with the suggestion that JAM run an Ethereum full node and the above, which is truly ostentatious! This is from > https://vitalik.eth.limo/general/2024/10/26/futures5.html Optimistic to the max, as usual.

2024-10-30 09:59 gav: Definitely reads like it’s written by a mathematician and not an engineer.

2024-10-30 09:59 gav: But if you mean the bottom bit, yes it’s an idea. I don’t see it being readily implementable from what is written here. It is however something that Parity is working on within the smart contracts team. (edited)

2024-10-30 09:59 gav: I don’t see much in there except half-baked ideas and a general lamenting of the realities of building and maintaining real systems (rather than mathematical constructs). (edited)

2024-10-30 11:19 dakkk: Should usage of LLM and similiar be allowed in JAM implementations? I think that since they are trained by the same data, they may introduce the same bug across multiple implementations

2024-10-30 11:56 gav: What do you mean “in JAM implementations”?

2024-10-30 11:56 dakkk: graypaper implementations*

2024-10-30 11:57 gav: By implementers to write code? Or by the code itself to govern its behaviour?

2024-10-30 11:57 dakkk: by implementers to write code using those tools

2024-10-30 11:58 gav: No. LLMs shouldn’t generally be used to write code to be submitted. This would count as collusion. (edited)

2024-10-30 12:01 dakkk: ok 👍️ it was already my opinion, but I asked since it is not specified in JAM prize rules.

2024-10-31 23:30 ycc3741: Hi I can't find my country in JAM Prize Interest Form

2024-10-31 23:30 ycc3741: I'd already sent the mail to jam@web3.foundation and might need so help.

2024-11-01 04:17 keegan: Thanks ycc3741 just responded

2024-11-02 07:24 sourabhniyogi: sub0 schedule is up! 💓 😍 🥰 https://agenda.sub0.gg/sub0-reset-2024/schedule/#2024-11-11 Nov 11th is packed with JAM talks (or closely related topics) on the Main Stage, but also plenty of time and space to meet each other over lunch/coffee. Registration is at: https://sub0.gg/

2024-11-02 07:59 gav: Do we have somewhere to congregate on the 9th/10th?

2024-11-02 08:15 dakkk: > <@sourabhniyogi:matrix.org> sub0 schedule is up! 💓 😍 🥰 > > https://agenda.sub0.gg/sub0-reset-2024/schedule/#2024-11-11 > > Nov 11th is packed with JAM talks (or closely related topics) on the Main Stage, but also plenty of time and space to meet each other over lunch/coffee. > > Registration is at: https://sub0.gg/ will it be available a live stream?

2024-11-04 03:14 jam_man: Are we still accepting teams that work in python? I’m a PhD candidate in computation neuroscience and have a team interested in working on a python implementation.

2024-11-04 03:17 jam_man: > <@jam_man:matrix.org> Are we still accepting teams that work in python? I’m a PhD candidate in computation neuroscience and have a team interested in working on a python implementation. I submitted an official application already. I am just wondering while we wait to hear back.

2024-11-04 03:41 jam_man: Second question. How many milestones / how many rewards are left to be paid out. I don’t see an easy place to see that. Thanks !

2024-11-04 03:42 xlchen: GP is not yet finalized so it is not yet currently for anyone to deliver any milestones

2024-11-04 03:43 jam_man: Thank you. Sorry but could you clarify what a GP is ?

2024-11-04 03:43 xlchen: graypaper

2024-11-04 03:43 jam_man: Gotcha thank you.👍

2024-11-04 03:44 xlchen: here are the participating teams: https://graypaper.com/clients/

2024-11-04 03:44 jam_man: Does that mean teams have not begun working yet? Or just that submission are halted until a finalized GP is out.

2024-11-04 03:46 xlchen: not everything are defined but many components are stable and people are working on those

2024-11-04 03:47 xlchen: https://github.com/gavofyork/graypaper this have bit more details

2024-11-04 03:47 jam_man: Are new teams still being accepted. Or is that list of clients you sent solidified

2024-11-04 03:47 xlchen: there is no acceptances, anyone are welcome to start a new implementation at any time

2024-11-04 03:48 xlchen: the list I shared is just a list of teams sharing they are participating. it is most likely not the complete list

2024-11-04 03:49 xlchen: the only thing matters is deliveries. e.g. no prize will be reserved for any teams

2024-11-04 03:51 jam_man: okay awesome thank you.

2024-11-04 06:49 dakkk: > <@jam_man:matrix.org> Are we still accepting teams that work in python? I’m a PhD candidate in computation neuroscience and have a team interested in working on a python implementation. There are other two teams working using python here, you are in good company ;)

2024-11-04 06:51 jam_man: Exciting! What are the rules on how many python implementations can get rewards for each milestone, I don’t see any rules on that.

2024-11-04 06:51 jam_man: Could all 3 python teams get rewards for all 5 mile stones?

2024-11-04 06:51 xlchen: https://jam.web3.foundation/rules

2024-11-04 06:52 jam_man: Ahh great thanks

2024-11-04 06:52 jam_man: Was looking for something that extensive 👍

2024-11-04 06:55 jam_man: Oh actually I have seen / read that before. I didn’t see any clarification on how many times a specific language could produce an implementations. I know one team can only do one implementation for each milestone. But hypothetically could all the rewards go to teams that happen to be working on python.

2024-11-04 06:57 xlchen: it is first-come, first-served so yes if for some reason all the python teams are faster than other teams in the same language set

2024-11-04 06:58 jam_man: Thank you good to know

2024-11-04 07:01 jam_man: I assume the answer to this first question is no. But Is it considered colluding to join a team that has already been working. 
Secondly would it be considered colluding or against the rules to have communication with a teams even if it simply the prospect of joining their team. (edited)

2024-11-04 07:05 jam_man: Colluding I assume no to either. But would either of these need to be officially disclosed.

2024-11-04 07:06 xlchen: not sure if there is written rule about those situation but IMO, teams can definitely be able to grow. I guess the only thing that's not allowed is to having people transferring between two teams

2024-11-04 07:08 jam_man: Okay yeah, i assumed once two teams had already begun working the transfer of members between groups would be prohibited. But growing with new independent members was fine.

2024-11-04 16:20 emielsebastiaan: > <@jam_man:matrix.org> I submitted an official application already. I am just wondering while we wait to hear back. I sent you a DM on X.

2024-11-05 02:13 xlchen: https://github.com/zdave-parity/jam-np/blob/main/simple.md#ce-129-state-request I don't really understand about the boundary node part. can we have some examples? also examples for each message request/response will be super helpful

2024-11-05 15:02 dave: > <@xlchen:matrix.org> https://github.com/zdave-parity/jam-np/blob/main/simple.md#ce-129-state-request > I don't really understand about the boundary node part. can we have some examples? also examples for each message request/response will be super helpful As an example, say the trie looks like this: a b c d e f g h i j k l m n o And we get a query where the start key matches leaf i and the end key matches leaf k. We would return the key/value pairs from i,j,k. We would also return a,b,d,i,e,k as boundary nodes. These boundary nodes cover the paths from the root (a) to the start and end keys (leaves i and k). Note that there is quite a bit of redundant information returned here; more than necessary to prove presence of i,j,k in the trie. This might be changed in the full protocol; it's defined this way to hopefully make the boundary proof easy to build and verify. If we get a query where the start key is _not_ in the trie, it is a bit more complicated as the boundary nodes need to cover the path where this key _would_ be found, if it was in the trie (in order to prove absence). If we imagine, for example, the same query but with the start key changed so it is greater than i's key but would still be found at the location of i in the trie. In that case, we would only return the key/value pairs from j and k, but the returned boundary nodes would be the same (a,b,d,i,e,k).

2024-11-05 15:03 dave: Example requests/responses for the various protocols will probably be covered by test vectors at some point, at least where this makes sense. These don't exist yet though.

2024-11-05 16:27 jaymansfield: Hello all. We have been silently coding away but are happy to finally share our first public update here. We are working on a Java implementation of JAM, temporary name is JavaJAM. Currently we have the following sets of test vectors passing: codec, safrole, disputes, recent history, trie and as of today PVM tests are also all passing now. Next priority is networking so we can start or join a testnet. Let's keep building! (edited)

2024-11-06 00:17 jam_man:

2024-11-06 02:45 bmaas: JamZig has landed at the Valia Hotel, Bangkok. Milestone #1: find koffie! ☕ (edited)

2024-11-06 12:28 sourabhniyogi: Does anyone have a Rust QUIC implementation demonstration showing the JAMSNP UP 0 handshake between Rust <> Rust? We can snowball a Go equivalent to match and show Rust <> Go (or whomever can contribute their own language), and I imagine everyone else can basically use this as a reference so JAMSNP connectivity issues are basically on par with FFIing into w3f's bandersnatch/ring-vrf+bls libraries (consistent with Rule 1).

2024-11-06 12:29 sourabhniyogi: We have some guesses as to how Ed25519 public keys are mapped into validator indexes but aren't super sure that our guesses are right. So, having this mapped out in a Rust <> Rust working demo would make everything crystal clear.

2024-11-06 12:39 dave: > <@sourabhniyogi:matrix.org> We have some guesses as to how Ed25519 public keys are mapped into validator indexes but aren't super sure that our guesses are right. So, having this mapped out in a Rust <> Rust working demo would make everything crystal clear. The index of a validator is simply its position in the validator key list in the state. Is this what you weren't sure about or something else?

2024-11-06 12:50 sourabhniyogi: > <@dave:parity.io> The index of a validator is simply its position in the validator key list in the state. Is this what you weren't sure about or something else? By this I mean if some node X connects to another node Y, node Y has to figure out which validator index X in the state it is. Both X and Y have some ed25519 pub key, which are used in the TLS handshake with self-signed certs as per https://github.com/zdave-parity/jam-np/blob/main/simple.md#encryption-and-handshake -- so both X and Y use their [genesis] state -- something like https://github.com/jam-duna/jamtestnet/blob/main/traces/safrole/genesis.json#L81-L118 which use develop -- to basically map the pub key from the cert into a validator index in that state. That's the first part, which gets all the basic niceties of TLS mapping into validator indexes out of the way Then the second part is everything here down to the exact "Unique Persistent" part https://github.com/zdave-parity/jam-np/blob/main/simple.md#up-0-block-announcement where we can just have some dummy data to support the Handshake and Announcement, maybe like every 6 seconds.

2024-11-06 12:52 sourabhniyogi: We can post our Go <> Go but I don't want to spread the wrong answer if someone has a Rust <> Rust correct answer?

2024-11-06 12:54 charliewinston14: Are third party libraries allowed for the QUIC protocol? The rules only mention crypto related ones but not all languages have it built in yet and would be a massive undertaking to write from scratch.

2024-11-06 12:55 dakkk: image.png

2024-11-06 13:03 sourabhniyogi: > <@charliewinston14:matrix.org> Are third party libraries allowed for the QUIC protocol? The rules only mention crypto related ones but not all languages have it built in yet and would be a massive undertaking to write from scratch. Definitely as per Rule 1 "Third-party libraries for cryptographic primitives (erasure-coding, Bandersnatch, Ed25519), codecs (e.g. SCALE), and networking (e.g. QUIC) are acceptable."

2024-11-06 13:04 sourabhniyogi: Has anyone succeeded with FFIing into one, just curious?

2024-11-06 13:06 dave: FWIW we're using Quinn at the moment in PolkaJam (Rust)

2024-11-06 13:10 sourabhniyogi: Ok for Quinn is there some snippet of rust that if we add to https://github.com/quinn-rs/quinn/blob/main/quinn/examples/server.rs#L168 gets at the Ed25519 pub key correctly? And covers the generation of the self-signed certs. (edited)

2024-11-06 13:17 dave: Once you have a Connection in Quinn you can call handshake_data to get at the peer's certificate (https://docs.rs/quinn/0.11.5/quinn/struct.Connection.html#method.handshake_data)

2024-11-06 13:28 dave: > <@dave:parity.io> Once you have a Connection in Quinn you can call handshake_data to get at the peer's certificate (https://docs.rs/quinn/0.11.5/quinn/struct.Connection.html#method.handshake_data) Ah sorry the function is peer_identity, not handshake_data

2024-11-06 13:32 dave: You should also set custom certificate verifiers for rustls to check the requirements given in the SNP doc. The requirement that the "alternative name" be derived from the public key was intended to fit with APIs that allow you to specify an expected name when connecting. eg In Quinn, Endpoint::connect takes an IP address and an expected name (https://docs.rs/quinn/0.11.5/quinn/struct.Endpoint.html#method.connect)

2024-11-06 13:38 sourabhniyogi: Got it -- It is sufficient to check the pub key, and ignore checking that the ip address in the metadata matches, right? A latent background question is: why do we need/want ip address data on-chain? (edited)

2024-11-06 13:39 dave: There is no need to verify IP address/port no

2024-11-06 13:39 dave: These are included on-chain so that all validators know how to connect to each other

2024-11-06 13:40 dave: In Polkadot there is a DHT for this, which has some advantages (eg IP/port can be changed at any time), but overall we decided it was not worth the extra complexity

2024-11-06 13:52 sourabhniyogi: Ok, how are validators expected to change their ip/port -- isn't this a slow process? is allowing dns into the mix a reasonable move for the ip part? It sure seems too easy to firewall out the entire network. Probably a rabbit hole discussion to be saved for another day!

2024-11-06 14:16 dave: A new validator set can be specified every epoch, with new addresses for all the validators in the set. New validator sets are queued, but epochs are only an hour each, so a transition to a new IP address should take at most a few hours. There is at least one wrinkle in this: exported segments are expected to be available for 28 days. The validators holding them may not be elected as validators again, and so may not get a chance to "announce" new addresses. In all honesty I'm not totally clear on what the incentives are for these validators to remain online and continue serving requests for the old exported data.

2024-11-06 14:17 dave: Not sure what you mean by "It sure seems too easy to firewall out the entire network"

2024-11-06 14:18 dave: Re allowing DNS, don't have a strong opinion on this, though it would be less simple :)

2024-11-06 17:30 xlchen: there is little needs to change ip address with ipv6

2024-11-06 17:31 xlchen: we are using msquic btw and it offers callback to perform custom cert validation

2024-11-07 02:05 jaymansfield: I have a question relating to JAMNP. For the ephemeral encryption key for the DH key exchange, is this also Ed25519? Or does it use TLS 1.3 default sig algorithms? (edited)

2024-11-07 02:19 xlchen: our implementation just use the default

2024-11-07 08:55 dave: You can just use the default settings for this, IIRC there is some negotiation to pick an algorithm both sides support

2024-11-07 13:14 emielsebastiaan: > <@gav:polkadot.io> Do we have somewhere to congregate on the 9th/10th? Did anyone share any new details for the 9th or 10th on any gathering of implementer teams? Lmk 🙏🏻

2024-11-07 14:21 oliver.tale-yazdi: > <@emielsebastiaan:matrix.org> Did anyone share any new details for the 9th or 10th on any gathering of implementer teams? Lmk 🙏🏻 i think this is the most up-to-date link https://polkadot.subsquare.io/referenda/1024

2024-11-08 01:54 sourabhniyogi: For the 9th/10th whoever gets there tomorrow first can find a spot/room and announce that spot for JAM builders here? For the 11th-16th I updated https://polkadot.subsquare.io/referenda/1024 with food arrangement details. Happy travels all (edited)

2024-11-08 10:54 ycc3741: Hi. I consider that I need some help. I'd sent the mail to your team and asking for whether or when our team can be on the JAM Prize client page. Look forward to your response, I appreciate your efforts. Thanks a lot.

2024-11-08 11:19 gav: > <@ycc3741:matrix.org> Hi. I consider that I need some help. I'd sent the mail to your team and asking for whether or when our team can be on the JAM Prize client page. Look forward to your response, I appreciate your efforts. Thanks a lot. There is a link for getting in touch with the W3F prize team on the W3F JAM Prize page

2024-11-08 11:20 gav: That link doesn't seem to mention 9-10. Is there any info on where and who will attend on the 9-10? (edited)

2024-11-08 17:00 olanod: Random thought, perhaps off-topic...I was very briefly checking hyperlight and their "micro vms" concept, it looks interesting and familiar to what we see with WASM runtimes or PolkaVM. Assuming 99% of chain infrastructure runs on x86_64 linux, would it be reasonable to support chain infrastructure running sandboxed binaries of the same architecture as the host machines?("polkavm micro vms" should still be possible).

2024-11-08 17:19 gav: Firstly, PVM is already pretty close in ISA to x86_64, by design, so x86 code should not be too hard to express as PVM. The harder part is making the I/O work - CPUs enjoy a much greater memory/disk random access bandwidth than blockchains due to the need to ferry data and prove correctness to all nodes involved in verifying. Secondly, the really hard part for a trustless ISA is gas accounting. A lot more becomes possible with effective secure deterministic metering and hyperlight doesn’t make any efforts to support this afaict. (edited)

2024-11-08 17:25 olanod: yeah I don't think metering is in their radar and might not be trivial to implement.

2024-11-09 04:46 ycc3741: > <@gav:polkadot.io> There is a link for getting in touch with the W3F prize team on the W3F JAM Prize page Thanks a lot. I'd sent the.

2024-11-09 04:46 ycc3741: I'd send the pr.

2024-11-09 04:47 gav: Maybe it’s worth setting up a group for those who will be in Valia Hotel over the next days?

2024-11-09 04:47 gav: @emielsebastiaan:matrix.org @sourabhniyogi:matrix.org?

2024-11-09 04:50 boymaas: > <@gav:polkadot.io> Maybe it’s worth setting up a group for those who will be in Valia Hotel over the next days? Great idea! Feel free to add me too.

2024-11-09 04:51 sourabhniyogi: > <@gav:polkadot.io> Maybe it’s worth setting up a group for those who will be in Valia Hotel over the next days? Can you be at valia after sub0 on 11th (you have the last talk) or on 12th ?

2024-11-09 04:57 gav: As I have previously said, I’ll be in Thailand 9th and 10th only.

2024-11-09 04:57 oliver.tale-yazdi: > <@gav:polkadot.io> Maybe it’s worth setting up a group for those who will be in Valia Hotel over the next days? kianenigma: and me are also in the area

2024-11-09 04:57 oliver.tale-yazdi: Currently sub0

2024-11-09 05:07 emielsebastiaan: I am around (sub0) and available. Same for danicuki

2024-11-09 05:17 gav: JAM0 chat: https://app.element.io/#/room/#jam0:polkadot.io

2024-11-09 06:01 gav: No -in person but I’ll be leaving shortly afterwards. 9th and 10th are my only full days

2024-11-09 09:59 emielsebastiaan: We’ll have a JAM meetup for JAM Implementers in private meeting room (Meeting Room II) with beamer at Valia Hotel from 19:00h - 22:00h tonight (Valia Hotel Bangkok, Sukhumvit 24). We’ll have plenty of pizza and drinks. 

We can accommodate ~25 people.
Agenda: 1. Team Introductions.
We’d appreciate a 5-10 minute presentation about your team, reasons for implementing JAM, and some lessons learned. 2. Graypaper Questions and Answers 3. Planning, timelines, versions, milestones. 4. Suggestions for additional topics are welcome. Please confirm your attendance. One more important fact: Gavin will be joining.

2024-11-09 10:01 emielsebastiaan: Join chat to confirm attendance: https://app.element.io/#/room/#jam0:polkadot.io

2024-11-09 11:00 emielsebastiaan: I know it is a last minute call. But if you or your team is implementing JAM please join our little meetup. It is a great opportunity to get to know each other and converge on some common topics. We hope to see you in one hour.

2024-11-09 11:20 emielsebastiaan: Meeting Room 2 can be found on the 18th floor and should be accessible by elevator. (edited)

2024-11-09 15:46 emielsebastiaan: ima_b848e0f.jpeg

2024-11-09 15:46 emielsebastiaan: ima_0087e61.jpeg

2024-11-09 15:46 emielsebastiaan: ima_d2fd7ea.jpeg

2024-11-09 15:46 emielsebastiaan: ima_d2c6157.jpeg

2024-11-09 15:46 emielsebastiaan: ima_4388d2e.jpeg

2024-11-09 15:46 emielsebastiaan: ima_ec8e5b7.jpeg

2024-11-09 15:47 emielsebastiaan: ima_24c082c.jpeg

2024-11-09 15:47 emielsebastiaan: ima_1bc1af5.jpeg

2024-11-09 15:47 emielsebastiaan: ima_c0934de.jpeg

2024-11-09 15:47 emielsebastiaan: ima_7755b40.jpeg

2024-11-09 15:47 emielsebastiaan: ima_5020b59.jpeg

2024-11-09 15:47 emielsebastiaan: ima_d1dbc71.jpeg

2024-11-09 15:47 emielsebastiaan: ima_9aeb50b.jpeg

2024-11-09 15:47 emielsebastiaan: ima_28c47bb.jpeg

2024-11-09 15:47 emielsebastiaan: ima_4601e36.jpeg

2024-11-09 15:48 emielsebastiaan: Lesson for next time: group picture. Thank you everyone for participating!!! 🙏🏻🙏🏻🙏🏻

2024-11-09 15:49 emielsebastiaan: Does anyone have a picture of me on stage?

2024-11-09 15:53 emielsebastiaan: Apologies Kamil, I did not take a picture of you on stage.

2024-11-09 15:55 emielsebastiaan: We hope to share recordings of presentations in due time.

2024-11-10 14:28 clearloop: If it is still possible to join the public implementation )) I sent the form from the get in touch button and waited reply for months, just know it already started xd (edited)

2024-11-10 14:44 xlchen: there is no need to get anyone’s approval before you can start it

2024-11-11 03:47 oliver.tale-yazdi: Hey i added a tutorial for certificate generation, just in case if someone needs a straight forward way to crosscheck: https://docs.jamcha.in/advanced/networking/Certificates Also please LMK if this seems wrong to you, will try to use this for networking soon with other teams

2024-11-11 05:47 danicuki: Video with the JAM Roadmap to 1.0 https://www.youtube.com/watch?v=Jq8XBs1wPhg

2024-11-11 08:44 danicuki: I didn't know, but looks like someone already thought about running Doom on a blockchain and build it for Cardano. https://x.com/inputoutputhk/status/1824496224764039678?s=46&t=uDWflhDt4F1i47bMUn3yKA Does anyone can explain the differences on JAM compared to the Cardano solution?

2024-11-11 08:47 xlchen: a 5s look it is still transaction based, which is fundamentally different compare to JAM, which is transactionless. ie no need anyone to trigger the execution

2024-11-11 09:12 oliver.tale-yazdi: > <@danicuki:matrix.org> I didn't know, but looks like someone already thought about running Doom on a blockchain and build it for Cardano. https://x.com/inputoutputhk/status/1824496224764039678?s=46&t=uDWflhDt4F1i47bMUn3yKA > > Does anyone can explain the differences on JAM compared to the Cardano solution? Jan had a demo about this last year already: https://github.com/koute/polkadoom, so not sure who got the idea from whom

2024-11-11 09:16 emielsebastiaan: ima_ed13495.jpeg

2024-11-11 09:16 emielsebastiaan: https://canitrundoom.org/

2024-11-11 09:18 bkchr: > <@xlchen:matrix.org> a 5s look it is still transaction based, which is fundamentally different compare to JAM, which is transactionless. ie no need anyone to trigger the execution Yeah and they are just committing the frames to the chain, while with JAM Doom itself could on a core and you just send the inputs as transactions

2024-11-11 09:18 bkchr: Would actually be some funny game. Like these twitch games. Everybody controls the movement

2024-11-11 10:02 ycc3741: Hi I had sent the pr which is #56 and accepted by the bot. I wonder when the website will be update. Thanks a lot. Also, today's event on youtube is awesome and I really appreciate your effort.

2024-11-11 18:26 danicuki: I have a conceptual question about JAM: does JAM offer any guarantee of privacy for users? Wouldn't be possible that the validators node owner have access to the hardware memory and could see everything that is being executed in their own node, unencrypted?

2024-11-11 18:47 tebdev: Good question and observation danicuki ! I have had clients that chose Corda and Hyperledger for this reason and others rolled their own blockchain e.g. JP Chase. Also does the Validator have a transport confidentiality requirement that justifies use of TLS? That then leads us to key lifecycle questions. (edited)

2024-11-11 20:36 dave: TLS is primarily for peer authentication and ensuring data integrity. The privacy aspect is not particularly important though it does matter in some cases (e.g. Safrole tickets would not be anonymous if everyone could see where they came from)

2024-11-11 20:42 dave: Re privacy within JAM, everything is public so anyone can see what is being computed. It would presumably be possible to have private computation in a service using ZK stuff though

2024-11-11 20:49 tebdev: > <@dave:parity.io> TLS is primarily for peer authentication and ensuring data integrity. The privacy aspect is not particularly important though it does matter in some cases (e.g. Safrole tickets would not be anonymous if everyone could see where they came from) Thank you for clarifying. If I read this right the secuirty requirements are 1) authentication 2) data integrity (transport versus payload(Tx, Extrensic) ? ). Can you help me understand risk of Safrole ticket confidentiality requirement?

2024-11-11 20:58 dave: One of the goals of Safrole is to keep the identity of the selected block authors secret up until the point they actually author their blocks, to prevent them being DoSed for example. Tickets themselves are anonymous due to the RingVRF magic, but it is also important that the way tickets are distributed does not reveal their author. This is achieved by first sending each generated ticket to a randomly selected "proxy" validator who then distributes the ticket more widely. Anyone who sees this ticket get sent to the proxy validator can infer who created it so the sending should be private. Of course the proxy validator could itself be compromised, this is hopefully rare!

2024-11-11 21:00 dave: > <@tebdev:matrix.org> Thank you for clarifying. If I read this right the secuirty requirements are 1) authentication 2) data integrity (transport versus payload(Tx, Extrensic) ? ). Can you help me understand risk of Safrole ticket confidentiality requirement? Re security requirements, for the transport between validators we definitely need authentication and integrity, otherwise the security arguments for the protocols built on top will fall apart

2024-11-11 21:11 tebdev: Please accept my apology in advance if I am too direct, The prize is not my primary incentive. I am trying to convince my stakeholders that Polkadot and JAM is the future. I ask questions to defend my Polkadot lemma. Very interesting ... "things fall apart" ... So Validator requires Trust to enable Trustless transactions. Does this indicate that Polkadot will become a CA since a key pair at server level a dependency for mTLS? (edited)

2024-11-11 21:15 dave: I meant only that if a validator thinks it's talking to another validator but is actually talking to someone else then that is bad

2024-11-11 21:15 dave: Don't think this is anything special to JAM

2024-11-11 21:16 tebdev: Agreed, but it seems to be a foundational element If not there may be confusion in the craft or my observation.

2024-11-11 21:18 tebdev: Please excuse me being a Polkadot neophite, but for one server to trust another keys, there must be trusted third party in a brokered trust model ... Right ?

2024-11-11 21:46 dave: If you identify servers by name then yes, but we just identify them by their public key

2024-11-11 21:46 tebdev: a public key is an identity as in authentication ... Right

2024-11-11 21:48 tebdev: In laymen terms public key (ID), private key (password) ... figuratively speaking

2024-11-11 21:49 tebdev: Ok the Server (1) is first party Server (2) is second party and Polkadot is trusted third party issueing keys to registered agents ... Private CA like Apple

2024-11-11 21:50 tebdev: In my humble view Polkadot is a private CA with all the rights, privileges and risk there unto appertaining ... ;)

2024-11-11 21:52 tebdev: Passport or a coin (staking) is just a hurdle any CA can legislate but in the end it is the same. trusted third party issuing trust with keys.

2024-11-11 21:58 tebdev: This is my humble Polkadot lemma . I am open to "reductio ad absurdum" ;), encourage it, welcome it because security a team quality sport. (edited)

2024-11-11 22:10 dave: > <@tebdev:matrix.org> Ok the Server (1) is first party Server (2) is second party and Polkadot is trusted third party issueing keys to registered agents ... Private CA like Apple No, validators generate their own keys. There is no trusted third party or central authority

2024-11-11 22:12 tebdev: OK, then validator creates own keys, server (1) and server (2) , which is reasonable, but what is mechanisms for them to trust each other ... JAM/Polkadot

2024-11-11 22:12 dave: Unless you consider the blessed chain that can set the validators to be a central authority, which I can see

2024-11-11 22:12 tebdev: Yes, I used term issue, but registration is another option for a CA

2024-11-11 22:13 tebdev: You still roll up to brokered trust model in the end, just semantics on who and what you call it.

2024-11-11 22:14 tebdev: Crypto chicken and egg... ;)

2024-11-11 22:15 tebdev: Google app store uses the CA registration versus Apple which is issuer ...

2024-11-11 22:15 tebdev: Both valid, but tradeoffs,

2024-11-11 22:19 tebdev: Polkadot as a Certificate Authority: Since Polkadot is issuing or registering keys for the servers, it qualifies as a certificate authority (CA) if these keys are used to establish mutual trust. The registration or issuance process is still central to Polkadot being considered a CA. In set notation, the formal definition becomes:

2024-11-11 22:19 tebdev:

2024-11-11 22:22 tebdev: image.png

2024-11-11 22:23 tebdev: If JAM/Polkadot will leave trust to the validators then OK, that is the law... If not then the judge is JAM/Polkadot

2024-11-11 22:33 tebdev: Net/Net, I need some help with this as foundational element to QUIC Key Life Cycle.

2024-11-11 23:12 dave: > <@tebdev:matrix.org> sent an image. Don't understand this sorry 😅

2024-11-11 23:26 dave: > <@tebdev:matrix.org> Net/Net, I need some help with this as foundational element to QUIC Key Life Cycle. Not sure what the question is exactly, sorry. You can argue that the blessed service is a central authority because it decides the next validator set. But assuming you trust the previously elected validator set, this service just behaves according to its code, which in practice will elect validators based on the staking decisions of Dot holders; there is no central authority.

2024-11-11 23:27 xlchen: I don’t really understand the question but there is some kind of chain of trust for validator keys. The root is genesis validators. And then via beefy/grandpa anyone can validate if a block is sealed by a current validator by proofs and exposed identity will be validator keys, which include ipv6 address in metadata

2024-11-11 23:27 tebdev: Sorry , early and multiplexing. Here is a better explanation.

2024-11-11 23:28 xlchen: the only centralized part is the genesis, which is possible to find someways to generate it in decentralized way

2024-11-11 23:30 tebdev: PolkadotasCA.pdf

2024-11-11 23:33 tebdev: > <@dave:parity.io> Not sure what the question is exactly, sorry. You can argue that the blessed service is a central authority because it decides the next validator set. But assuming you trust the previously elected validator set, this service just behaves according to its code, which in practice will elect validators based on the staking decisions of Dot holders; there is no central authority. Good point, but staking is a gate that Polkadot can up/down vote . If Validator approved how does the QUIC get a key it will trust. This goes back to your "fall down" comment to avoid a bad actor/imposter

2024-11-11 23:35 tebdev: Even if we have Governance and voting, the governance body = Polkadot

2024-11-11 23:35 tebdev: Or better said Polkadot stakers

2024-11-11 23:55 dave: To answer "If Validator approved how does the QUIC get a key it will trust": the blessed service provides the keys when specifying a new validator set

2024-11-11 23:59 tebdev: Ok , fair enough ... NO arguments from me, trying to walk it down to get my head around it, Blessed Service = (Some Polkadot Community) = (Distributed Authority) --> Issue keys (X.509, EC, Dual Key Pair) = Certificate. So would an entity issuing certificates with some rules or authority be a CA?.

2024-11-12 00:19 dave: Not sure what you mean by (X.509, EC, Dual Key Pair) = Certificate. Each validator has 2 public keys at the moment: an Ed25519 public key and a "Bandersnatch" public key. These are the keys that are provided when the blessed service specifies a new validator set. The nodes generate their own certificates containing their Ed25519 key. This is done only to keep the TLS protocol happy. The certificates are self-signed but these signatures are not expected to be checked. FWIW I wouldn't say the authority "issues" keys but rather selects keys that have been previously generated by nodes.

2024-11-12 00:23 tebdev:

2024-11-12 00:28 tebdev: Ok, corrected... I was being figurative in speech to say a dual key pair of a mutually agreed type. I may have got ahead of myself with "Certificate." If I am validator and I can be slashed, I would want a trust chain for this key signed and sealed by whomever sent it to me e.g. a "Certificate" ... This could be a Tx Extrinsic signature or a self contained root chain like X.509. QUIC and TLS best practice is X.509 to handle numerous risk scenarios.

2024-11-12 00:31 tebdev: This can also be genesis related as Bryan Chen | Laminar & Acala mentioned to establish a trust chain.

2024-11-12 00:32 tebdev: The validator would check public key to whatever trust chain before assuming trust of another validator?

2024-11-12 00:33 xlchen: what do you mean by check publicly key? they are read from onchain state which are checked

2024-11-12 00:34 tebdev: This delegates risk of bad key to governing body or genesis to mitigate validator risk

2024-11-12 00:34 xlchen: the risk of a few bad keys are minimal. that’s what PoS about (which is not part of GP)

2024-11-12 00:36 tebdev: Well, in all my security training we can 1) accept risk, 2) defer risk 3) mitigate risk ... Just need know which one is being asserted.

2024-11-12 00:36 tebdev: I am easy ;)

2024-11-12 00:37 tebdev: The flip side, I must be transparent to stakeholders when I explain. Then they chose to accept risk (edited)

2024-11-12 00:38 xlchen: GP implicitly covers the part of the maximum damage can be done by few bad keys

2024-11-12 00:38 xlchen: ie not possible to have bad reports to be finalized

2024-11-12 00:40 tebdev: Fair enough, can you provide document. I am easy, but those that care will ask where did key come from and how can we trust it? If we just defer / accept risk , then that must be reveiwed for indemnification.

2024-11-12 00:40 xlchen: the risks are mitigated

2024-11-12 00:41 tebdev: OK, I am novice, help me understand

2024-11-12 00:41 xlchen: it is not possible for example to finalize a bad parachain block currently in polkadot 1.0. same for JAM

2024-11-12 00:42 tebdev: Hmm, I think we comparing apples to oranges... ;)

2024-11-12 00:44 xlchen: they are same. there are auditing process defined by GP to ensure bad reports will be identified and reported and having bad validators to be removed

2024-11-12 00:48 tebdev: I am only focused on QUIC server (1) <--> Server (2) TLS handshake now. Respectfully, I think what you referring to comes lager ...

2024-11-12 00:49 xlchen: then the max damage is dos, which is nothing

2024-11-12 00:50 xlchen: I am talking about max damage of a bad key

2024-11-12 00:50 xlchen: there is little damage can be done on network level

2024-11-12 00:52 xlchen: of course the node must implement the usual networking best practice. ie rate limiting, peer reputation, message sanitation etc

2024-11-12 00:54 xlchen: it will be bad for example if everyone is using openssl with another heartbleed bug. and that’s why we should use different libraries (edited)

2024-11-12 00:59 tebdev: Key management with X.509 is an industry best practice when using TLS even if QUIC(TLS) ., We can decide not to do that, but needs to be a decision at some level or maybe backlogged as issue to keep moving forward.

2024-11-12 01:03 tebdev: > <@xlchen:matrix.org> it will be bad for example if everyone is using openssl with another heartbleed bug. and that’s why we should use different libraries I agree, but think we should discuss use of different sources. I developed "Aerospace" and Crititical infrastructure code... commented deciding to do clean room and funding it was Nobel Prize level for unstoppable. We should seek unstoppable for the network too.

2024-11-12 01:05 sourabhniyogi: For those of you not in Bangkok, this is from this weekend's JAM0 meetup with Gav https://www.youtube.com/watch?v=Jq8XBs1wPhg https://x.com/colorfulnotion/status/1856135098929951219 Its more detailed than the sub0 talk of yesterday, so JAM implementers won't want to miss it!

2024-11-12 01:15 tebdev: To be clear, I have no dog in fight, I am following Sr. Polkadot guys. I just want to know where dogs may fight to avoid it ;)

2024-11-12 01:17 tebdev: Maybe we agree QUIC keys are either issued or registered and tied to some authority to establish a trust chain and put in backlog as issue. ... I am flexible.

2024-11-12 01:19 xlchen: we are not going to make adjustments just because some people may have false understanding of security. we can provide more education content to explain why our approach is secure but if people insist on useless addition, well, I don’t know, I am not going to make my life harder because of that

2024-11-12 01:20 xlchen: there IS a chain of trust already

2024-11-12 01:20 xlchen: it is not like people can become validators as they want

2024-11-12 01:21 dave: SNP uses certificates only because this is the path of least resistance: basically all QUIC libraries are designed for the Web 2.0 use case of a client connecting to a server via a name like google.com. In that case you need a certificate chain and trusted root to establish a link between google.com and a public key. In the case of JAM, we start with a public key so the certificate stuff is all somewhat pointless. It's just hoop jumping so that the protocol can be implemented without having to hack around with QUIC libraries too much. In a perfect world we would just use some noise protocol (http://www.noiseprotocol.org/) and there would be no certificates involved.

2024-11-12 01:21 tebdev: Fair enough. I did ask to help me understand as neophyte. I am too old to argue and to fat to fight. Also I am flexible and easy. I just ask for explanation so I can explain to others that rely on me.

2024-11-12 01:22 tebdev: NO one asked you to change. I do not know where you got that.

2024-11-12 01:22 tebdev: I hope this is safe place for open dialogue.

2024-11-12 01:23 tebdev: Security = Quality, Quality = teamwork, Teamwork = open mind

2024-11-12 01:24 xlchen: > <@tebdev:matrix.org> Maybe we agree QUIC keys are either issued or registered and tied to some authority to establish a trust chain and put in backlog as issue. ... I am flexible. this read like a change request? anyway, nothing personal. I don’t mind this kind of discussion. always happy to discuss things and I understand I maybe wrong as well

2024-11-12 01:24 tebdev: At the end it says "I am flexible" ;)

2024-11-12 01:26 tebdev: If you do not want something in backlog, I am OK. I just document and move on. I am truly easy...

2024-11-12 01:29 tebdev: I was asking for help to understand since I was not in the room when the design was developed. David was a good sport and made me think... this should be fun :)

2024-11-12 01:31 tebdev: No worries I am headed to Ethereum conference. I got people asking me about that one too. Maybe we catchup later with a beer. I have very open mind.

2024-11-12 03:49 danicuki: My initial question about security was exactly similar to Boy s condition: I am very happy to be building JAM. And can't keep this happyness within myself: I share what I am doing with others. People outside have many questions. I know how to answer most of them, but some are trickiest. That's why I ask here. We know some people are more specialists than me. I am like Boy, kind of new to Polkadot. One idea would be to maybe include a FAQ section in the graypaper website or any other place might be convenient.

2024-11-12 04:05 danicuki: If computation done in JAM is "public" - because hardware owner can read the memory - does this mean that some applications are not suitable for JAM, because of its design? Or am I missing something? (edited)

2024-11-12 06:54 emielsebastiaan: This is the point of having a public world computer. You can build private systems on a public infrastructure you cannot build a public system on a private infrastructure.

2024-11-12 07:09 tebdev: World Computer ... Indeed. Eth BT discussed that in keynote ... L1 = The Trust, L2 = The GPU ... Hmmm this sounds like Polkadot but with more validators

2024-11-12 07:10 tebdev: hmmm (edited)

2024-11-12 13:21 olanod: Any word or place to discuss on the capabilities of the vm(during refine at least), i.e. what syscalls/host-functions would be available to guest programs? is it going to be very limited, fixed and jam specific or is it something that could evolve over time? I like the approach of WASI, after its MVP that provided basic std support(e.g IO with file descriptors), it's evolving into a flexible "component model" where interfaces are defined with the WIT IDL and based on the platform different bundles("worlds") are defined, interfaces can also include higher level capabilities like http clients or ML primitives(e.g. wasi-nn). Could we do something similar or better yet just use the same stuff? I imagine the Polkadot fellowship could be in charge of discussing via RFCs what goes in this "jam world" and jam implementers would be expected to support the agreed on capabilities.

2024-11-12 17:06 gav: JAM is a trustless supercomputer. this implies that it's a global singleton. whatever it does must be universally agreed. this is why we go to great pains to write a single formal specification that exactly describes correct behaviour in any possible circumstance. (edited)

2024-11-12 17:08 gav: if you can find a way to express an http client (or indeed any real-world I/O) in such a way, feel free to make a PR to the graypaper. (edited)

2024-11-12 17:09 gav: but i think you'll find it's a fool's mission.

2024-11-12 17:10 gav: (this is because the correct behaviour of an HTTP request is totally dependent on factors which cannot be universally agreed including network status, timing, server response, physical interface characteristics, ...) (edited)

2024-11-12 17:11 gav: without doing a comprehensive analysis i'd expect that the same is true for pretty much all useful sys calls.

2024-11-12 17:12 gav: there may be some which expose computation machinery which can be universally agreed upon to be available (e.g. a particular amount of "GPU gas"). this could _potentially_ be an in-core resource down the line. (edited)

2024-11-12 20:23 charliewinston14: Question about JAMNP. For the preferred initiator equation the last part has just (a < b) where a and b are ed25519 keys. What is meant by that comparison?

2024-11-12 21:08 alxmirap: hi, in my understanding, this equation is the XOR of 3 booleans, or equivalently 1-bit values. a and b are 32-byte arrays, but in essence they can be thought of as very large numbers of the same size, so they can be sorted numerically. Then, a < b is literally their numerical comparison. Given that they are of the same domain, if they are both represented in, say, hex strings of the same size (as in, don't ignore leading 0s) then an alphabetical comparison should be equivalent to the numerical one.

2024-11-12 21:23 tebdev: > <@alxmirap:parity.io> hi, in my understanding, this equation is the XOR of 3 booleans, or equivalently 1-bit values. a and b are 32-byte arrays, but in essence they can be thought of as very large numbers of the same size, so they can be sorted numerically. Then, a < b is literally their numerical comparison. > > Given that they are of the same domain, if they are both represented in, say, hex strings of the same size (as in, don't ignore leading 0s) then an alphabetical comparison should be equivalent to the numerical one. agree with your statement, in C++ not a problem going from hex to int , If very large use Boost library ;)

2024-11-12 22:39 gav: > <@charliewinston14:matrix.org> Question about JAMNP. For the preferred initiator equation the last part has just (a < b) where a and b are ed25519 keys. What is meant by that comparison? a and b are 32 byte quantities. They can be compared using a standard dictionary comparator.

2024-11-12 22:55 tebdev: > <@gav:polkadot.io> a and b are 32 byte quantities. They can be compared using a standard dictionary comparator. Agreed... good point. Using a "dictionary" more efficient std::lexicographical_compare, but depending on language including C++ can just be... return hexStr1 < hexStr2, but do not want to speak for other languages.

2024-11-12 23:04 tebdev: > <@xlchen:matrix.org> this read like a change request? anyway, nothing personal. I don’t mind this kind of discussion. always happy to discuss things and I understand I maybe wrong as well Please excuse, philosophically I agree this is better served as PR to JAM YP to avoid being disruptive. I am new to this project.

2024-11-13 05:59 haikoschol: At the JAM0 meeting yesterday we discussed what kind of dev tools we might want. ltfschoen just described how he is using TDD to implement Safrole, adding comments to the tests that reference the relevant part of the graypaper. If those comments were in a specific format, a tool could identify changes to the graypaper that impact the behaviour those tests cover. Maybe the tool could use links to the fluffylabs reader (diffing HTML might be brittle though). Or it could use the LaTeX source. Does that sound reasonable/useful to anyone?

2024-11-13 08:50 leonidas_m: Hello, given that passing the state-transition conformance tests is required for Milestone 1, do we need to fully implement the validator-core assignment logic as detailed in Section 11.3, or can we temporarily mock this for now (M1)?

2024-11-13 08:54 dakkk: > <@leonidas_m:matrix.org> Hello, given that passing the state-transition conformance tests is required for Milestone 1, do we need to fully implement the validator-core assignment logic as detailed in Section 11.3, or can we temporarily mock this for now (M1)? g and g* are required for guarantees extrinsic validation, so you need to implement 11.3 in order to check the validity of a block

2024-11-13 17:12 tomusdrw: > <@haikoschol:matrix.org> At the JAM0 meeting yesterday we discussed what kind of dev tools we might want. > > ltfschoen just described how he is using TDD to implement Safrole, adding comments to the tests that reference the relevant part of the graypaper. > > If those comments were in a specific format, a tool could identify changes to the graypaper that impact the behaviour those tests cover. Maybe the tool could use links to the fluffylabs reader (diffing HTML might be brittle though). Or it could use the LaTeX source. > > Does that sound reasonable/useful to anyone? We are using the links to the reader heavily in our code (duh!) and I think this kind of tool would make perfect sense. I've passed that to the team and I think we can figure something out. There is no need to parse HTML, because the reader link contains synctex blocks which point exactly to the latex source line that contains the selection. So it's just a matter of checking if that particular latex source line has been changed between versions.

2024-11-14 07:00 sourabhniyogi: Should CE 129 have 31-byte keys in the response or 32-byte keys in the response? We believe it would be beneficial to have responses be not 31 bytes but instead 32 bytes, in order for a full range scan of CE129 (from 0x00...00 to 0xFF..FF) to have full 32-byte keys (and of course values) hashable into a comparable state root. Generating state roots with the 31 bytes, given the lossy service storage, is not possible -- but having 32 byte keys requires a tad more bookkeeping to match up with some UI like this https://trie.fluffylabs.dev/

2024-11-14 07:57 dakkk: There will be a video of the today JAM implementers meetup at Bangkok?

2024-11-14 09:28 tomusdrw: > <@sourabhniyogi:matrix.org> Should CE 129 have 31-byte keys in the response or 32-byte keys in the response? > > We believe it would be beneficial to have responses be not 31 bytes but instead 32 bytes, in order for a full range scan of CE129 (from 0x00...00 to 0xFF..FF) to have full 32-byte keys (and of course values) hashable into a comparable state root. > > Generating state roots with the 31 bytes, given the lossy service storage, is not possible -- but having 32 byte keys requires a tad more bookkeeping to match up with some UI like this https://trie.fluffylabs.dev/ IMHO it's better to just use 31-byte for everything. I don't see a reason for this extra byte being useful at all. Also I'd love to see the node hash truncated to 255bits everywhere to allow trie manipulation without recomputing the hash. If you merkelize everything from scratch, there is obviously no issue, but if you allow adding/removing nodes it's sometimes required to recompute the last bit in case the node moves from left to right branch.

2024-11-14 10:16 sourabhniyogi: > <@dakkk:matrix.org> There will be a video of the today JAM implementers meetup at Bangkok? We will have a second session on Friday at 11am and will share notes, but no recording :(

2024-11-14 10:17 dakkk: got it (y) I'm a bit confused about those meetups, aren't a potential occasion of violating collusion rule? (edited)

2024-11-14 11:16 emielsebastiaan: Videos for the Saturday meetup have been published. We mostly got to know each other a bit better and got an opportunity to ask questions with Gavin present. 
I was also present at the Tuesday meetup.
I can assure you we did not discuss anything that violates (collusion) prize rules. That said, I am no longer in Bangkok now. 
Discussion on Tuesday was mostly related to:

1. Awareness of tooling that is already present and useful for everyone (e.g. fluffylabs) 2. Creating a list of things that would be useful besides the implementations (such as telemetry, explorer, cli flags, docker, RPC, other uis, etc) 3. Some coordination/scoping of M1, M2 priorities in order to have some form of cross-client testnet as initiated by Colorful Notion with useful phases to get implementer teams on board. 4. Discussion on having a (private) monthly call for implementers to further discuss on these topics.
 It would be great if we can get notes out here on the chat. 1. To be transparent, 2. To follow up with actionable items. (edited)

2024-11-14 11:20 emielsebastiaan: Video’s Saturday: https://youtube.com/playlist?list=PLjGN96vX3BTKjLfBd5LimBuDuch-QfKPy&si=JyiUssgVrnaIdXqo Scroll down to the end of the playlist.

2024-11-14 11:37 emielsebastiaan: Notes by Oliver Tale-Yazdi - async and sourabhniyogi here: https://matrix.to/#/!KKOmuUpvYKPcniwOzw%3Amatrix.org/%24DL2Vsua2TFzSAZDzOYO4pH0A6Ud7cABS6C49vBZhx28

2024-11-14 17:32 prematurata: hey davxy , starting from the padded ring commitment commit i am having issues in generating posterior_gamma_z. I've seen you changed all gamma_zs so I find it hard to pinpoint what changed at that point in time. Can you help out?

2024-11-14 17:36 prematurata: > <@prematurata:matrix.org> hey davxy , starting from the padded ring commitment commit i am having issues in generating posterior_gamma_z. I've seen you changed all gamma_zs so I find it hard to pinpoint what changed at that point in time. Can you help out? ah i see ark-ec-vrfs also changed 2 months ago. Updating the underlying dependency fixed my issues. sorry for ping

2024-11-14 18:42 tomusdrw:

2024-11-14 18:42 tomusdrw: > <@tomusdrw:matrix.org> We are using the links to the reader heavily in our code (duh!) and I think this kind of tool would make perfect sense. > I've passed that to the team and I think we can figure something out. > > There is no need to parse HTML, because the reader link contains synctex blocks which point exactly to the latex source line that contains the selection. So it's just a matter of checking if that particular latex source line has been changed between versions. You ask, we provide: https://www.npmjs.com/package/@fluffylabs/links-check

2024-11-15 13:00 danicuki: Here is the video with JAM 1.0 detailed road map presented by Gav last week: https://youtu.be/Jq8XBs1wPhg?si=8KvEZdAyeqeqaYP7

2024-11-15 19:20 sourabhniyogi: We would like to get systematic-ish access to the toaster to test V=1023 in mid-December after spinning up medium sized kubernetes cluster in Google Cloud of around V=384. What do we do exactly?

2024-11-15 20:09 gav: Wait until it’s built, for a start

2024-11-15 20:09 gav: mid-December is likely to be around the right timing though

2024-11-15 21:48 sourabhniyogi: Are there some smaller toaster configurations / parameters that could be appropriately be considered the Kusama JAM chain (and Paseo JAM chain) or are they going to be the same V? (edited)

2024-11-15 21:56 gav: I’d default to the same size unless there’s good reason to change it. (edited)

2024-11-15 21:57 gav: As for the spec I’d have thought around 50% of the performance would be about right.

2024-11-15 22:22 sourabhniyogi: I was thinking slightly more than V=341 for Paseo and slightly less than V=682 for Kusama (which could fit into the V=1023 toaster hardware) so as to grow like 1-2-3 but if that means teams spend time on appendix H variations it might not be worth it.

2024-11-15 22:40 sourabhniyogi: In the big picture, is refine suitable for zk proof generation ? Are there any GPU resources per chance in this toaster we could use for some zk proof generation side use case?

2024-11-16 08:44 gav: > <@sourabhniyogi:matrix.org> In the big picture, is refine suitable for zk proof generation ? Are there any GPU resources per chance in this toaster we could use for some zk proof generation side use case? No and it’d be dumb anyway.

2024-11-16 08:45 gav: Refine is already trustless.

2024-11-16 08:45 gav: ZK proof generation is verifiable. That’s the whole point of it.

2024-11-16 15:18 ycc3741: Sorry, I would like to clarify whether we should mock the imported block to test the import functionality, or if there is any data already provided by Parity?

2024-11-16 16:04 davxy: Some vectors related to WP reporting STF (Section 11 - Part 2) are now available. As always, your feedback is most welcome. https://github.com/w3f/jamtestvectors/pull/20

2024-11-16 16:05 davxy: https://github.com/davxy/jam-test-vectors/blob/work-reports/reports/README.md

2024-11-16 22:27 gav: Rollups take time to get from sequencing through to fraud-detection-period. JAM gives certainty faster. This allows message/value transfers to happen almost instantaneously. JAM may or may not be competitive with ZK rollups. Difficult to tell since they’re currently vapourware.

2024-11-16 22:33 gav: Just that.

2024-11-16 22:34 gav: An actual, economically viable, production implementation which works with fully Turing complete computation on the level of a JAM core and doesn’t take several minutes to get from sequencing to finality. (edited)

2024-11-16 22:46 sourabhniyogi: For "JAM may or may not be competitive with ZK rollups" why not take an approach of "If you can't beat them, JOIN them" (pun intended 😅 , Join-Accumulate ZRKUs) by aiming to have the worlds ZKRU proofs indexed in JAM DA along with all ORU proofs of validity for rollups that cant afford ZK prover networks. It seems to be the winning approach?

2024-11-16 22:48 gav: Feel free to make whatever service you feel you can sell. (edited)

2024-11-17 18:29 rustybot: > <@prematurata:matrix.org> hey davxy , starting from the padded ring commitment commit i am having issues in generating posterior_gamma_z. I've seen you changed all gamma_zs so I find it hard to pinpoint what changed at that point in time. Can you help out? @[prematurata | TSJam] https://github.com/davxy/jam-test-vectors/blob/safrole-ring-signatures-update/safrole/README.md#%EF%B8%8F-warning--%EF%B8%8F

2024-11-19 06:42 sourabhniyogi: Pictures from JAM0 are up on Polkadot forum: https://forum.polkadot.network/t/jam0-jam-implementers-meetup-sub0-devcon-7-bangkok-nov-11-nov-16-2024/10866 If you enjoyed the event and want to do a similar event again in late Winter / early Spring 2025, please post something there, thank you all for making this a special week for everyone! (edited)

2024-11-19 06:53 sourabhniyogi: Please retweet https://x.com/colorfulnotion/status/1858765451071000716 =)

2024-11-19 16:04 danicuki: Useful PVM invocation dependency created by Luke | Jamixir

2024-11-19 16:04 danicuki:

2024-11-19 16:04 danicuki: image.png

2024-11-19 17:06 ycc3741: Pardon. Regarding Appendix D. In State Merklization, do we need to manually implement the serialization and encoding functions B and L ourselves, or are there available libraries for this?

2024-11-19 21:55 jaymansfield: > <@sourabhniyogi:matrix.org> Pictures from JAM0 are up on Polkadot forum: > > https://forum.polkadot.network/t/jam0-jam-implementers-meetup-sub0-devcon-7-bangkok-nov-11-nov-16-2024/10866 > > If you enjoyed the event and want to do a similar event again in late Winter / early Spring 2025, please post something there, thank you all for making this a special week for everyone! Wish we could have been apart of this!

2024-11-20 19:04 danicuki: Hi davxy, would you please explain what is this "entropy" in input? Shouldn't entropy be part of the pre_state? https://github.com/davxy/jam-test-vectors/blob/7dc0f5cee1552bd535a4cd76836b9d2a279fb878/reports/tiny/anchor_not_recent-1.json#L55

2024-11-20 19:16 greywolve: Isn't this because the entropy pool doesn't change, so it's just an input? (edited)

2024-11-20 20:17 danicuki: Actually I don't see why we can't simple use only pre-state and block as inputs. In the end, STF is just this: inputs are blocks and state. Would be much simpler than having another "input" which, in the end, must be transformed into some block and state to run the STF.

2024-11-20 23:48 xlchen: can we please having the PRs in the jamtestvectors merged? it is not fun to try to run tests against with all those branches. it is not like we can't update the tests after they are merged

2024-11-21 06:58 rustybot: > <@danicuki:matrix.org> Actually I don't see why we can't simple use only pre-state and block as inputs. In the end, STF is just this: inputs are blocks and state. Would be much simpler than having another "input" which, in the end, must be transformed into some block and state to run the STF. Right. I'll move it to the state

2024-11-21 07:29 sourabhniyogi: A test vector in the codec with a full JSON + JAM-codec for state (C1-C15 _and_ some service account 42) with a precise state root (to get the 31 vs 32 byte key detail resolved) would be terrific -- I imagine it should follow this: https://graypaper.fluffylabs.dev/#/911af30/358101358101

2024-11-21 09:45 gav: > <@xlchen:matrix.org> can we please having the PRs in the jamtestvectors merged? it is not fun to try to run tests against with all those branches. it is not like we can't update the tests after they are merged We're working on doing this now

2024-11-21 12:03 gav: JPEG image-46E2-99A7-AC-0.jpeg

2024-11-21 12:04 gav: JAM Toaster/Polkadot Palace internet uplink speed test. (edited)

2024-11-21 12:04 gav: Some other JAM Toaster pics...

2024-11-21 12:04 gav: JPEG image-4D01-A7E4-92-0.jpeg

2024-11-21 12:04 darkartistry: > <@gav:polkadot.io> sent an image. How far apart were the machines

2024-11-21 12:04 gav: JPEG image-498F-958B-74-0.jpeg

2024-11-21 12:05 gav: JPEG image-4EC7-BFE3-64-0.jpeg

2024-11-21 12:05 gav: > <@darkartistry:matrix.org> How far apart were the machines This is internet uplink.

2024-11-21 12:05 gav: > <@gav:polkadot.io> sent an image. Power distribution racks.

2024-11-21 12:05 gav: > <@gav:polkadot.io> sent an image. Insulated panels for JAM Toaster enclosures.

2024-11-21 12:05 gav: > <@gav:polkadot.io> sent an image. Installing the panels.

2024-11-21 12:06 darkartistry: What is the costs of maintaining this toaster ?

2024-11-21 12:06 darkartistry: 100k per month ? Based on the uplink speed …

2024-11-21 12:08 gav: Wouldn't know.

2024-11-21 12:08 gav: I don't think it's anything nearly as expensive as that though

2024-11-21 12:10 vinsystems: Amazing! 🤩

2024-11-21 12:12 gav: Data_Center_video.mp4

2024-11-21 12:12 darkartistry: The monthly cost of a 1 Gbps (Gigabit per second) connection in data centers varies based on several factors, including geographic location, service provider, and specific service requirements. Here’s an overview: Geographic Variations: • North America and Europe: In these regions, transit costs are relatively low. For instance, a 1 Gbps connection might be priced around $10 per Mbps per month, totaling approximately $10,000 monthly. However, actual prices are often lower due to competition and peering agreements.  • Asia: Transit pricing is significantly higher, approximately seven times that of North America and Europe. This means a 1 Gbps connection could cost around $70,000 per month. Peering arrangements can reduce this cost, but the effective price remains higher than in Western regions.  Service Provider Examples: • Dedicated Internet Access (DIA): For a 1 Gbps DIA circuit, prices can vary. A general guide suggests that any 500 Mbps+ DIA circuit priced under $1,000 per month is considered a solid deal, though this is typically outside data centers.  • Colocation Services: Colocation providers often offer bandwidth as part of their packages. For example, some providers include 1 Gbps connections in their colocation plans, with overall monthly costs ranging from $79 for a single server to $599 for a full rack. These prices encompass space, power, and bandwidth.  Additional Considerations: • Peering vs. Transit: Engaging in peering agreements can significantly reduce bandwidth costs compared to purchasing transit from Tier 1 providers. The extent of peering varies by region and provider. • Data Center Location: Data centers in regions with higher operational costs, such as Asia, may charge more for bandwidth. Conversely, facilities in areas with lower costs may offer more competitive pricing. • Service Level Agreements (SLAs): Premium SLAs with guaranteed uptime and performance may increase costs. In summary, while a 1 Gbps connection in data centers can range from under $1,000 to over $70,000 per month, the actual cost depends on various factors, including location, provider, and specific service requirements. It’s advisable to consult directly with data center providers to obtain accurate and tailored pricing.

2024-11-21 12:12 gav: And here's a simple 3d model of the final setup

2024-11-21 12:12 darkartistry: > <@darkartistry:matrix.org> The monthly cost of a 1 Gbps (Gigabit per second) connection in data centers varies based on several factors, including geographic location, service provider, and specific service requirements. Here’s an overview: > > Geographic Variations: > • North America and Europe: In these regions, transit costs are relatively low. For instance, a 1 Gbps connection might be priced around $10 per Mbps per month, totaling approximately $10,000 monthly. However, actual prices are often lower due to competition and peering agreements.  > • Asia: Transit pricing is significantly higher, approximately seven times that of North America and Europe. This means a 1 Gbps connection could cost around $70,000 per month. Peering arrangements can reduce this cost, but the effective price remains higher than in Western regions.  > > Service Provider Examples: > • Dedicated Internet Access (DIA): For a 1 Gbps DIA circuit, prices can vary. A general guide suggests that any 500 Mbps+ DIA circuit priced under $1,000 per month is considered a solid deal, though this is typically outside data centers.  > • Colocation Services: Colocation providers often offer bandwidth as part of their packages. For example, some providers include 1 Gbps connections in their colocation plans, with overall monthly costs ranging from $79 for a single server to $599 for a full rack. These prices encompass space, power, and bandwidth.  > > Additional Considerations: > • Peering vs. Transit: Engaging in peering agreements can significantly reduce bandwidth costs compared to purchasing transit from Tier 1 providers. The extent of peering varies by region and provider. > • Data Center Location: Data centers in regions with higher operational costs, such as Asia, may charge more for bandwidth. Conversely, facilities in areas with lower costs may offer more competitive pricing. > • Service Level Agreements (SLAs): Premium SLAs with guaranteed uptime and performance may increase costs. > > In summary, while a 1 Gbps connection in data centers can range from under $1,000 to over $70,000 per month, the actual cost depends on various factors, including location, provider, and specific service requirements. It’s advisable to consult directly with data center providers to obtain accurate and tailored pricing. These were why I estimated 100k

2024-11-21 12:13 gav: I'm not certain but I think it was more around 5k/mon

2024-11-21 12:14 gav: maybe quite a bit less

2024-11-21 12:14 darkartistry: Cooooool !!!

2024-11-21 12:14 gav: but i know they had to lay a new fibre line so it took a while to get installed

2024-11-21 12:14 darkartistry: Nice I’ll follow closely on the progress ! 🙇🏻‍♂️🙇🏻‍♂️🙇🏻‍♂️

2024-11-21 12:16 vinsystems: Is in Portugal?

2024-11-21 12:18 gav: Yes - at the Polkadot Palace in Cascais

2024-11-21 12:19 gav: the existing pool is used as a giant heat sink for the toaster

2024-11-21 12:19 gav: So the toaster is located in an underground concrete "bunker" next to the pool's filtration system

2024-11-21 12:22 subotic: cool 😀 good luck with the construction 🍀🍀🍀 I hope all goes smoothly.

2024-11-21 12:27 gav: me too:) aiming to have phase 1 complete EOY

2024-11-21 12:30 gav: that's 86 machines, each with 96 cores, simulating full JAM with 8 cores/node. it'll then be extended by 85 more machines (phase 2) so each node has 16 cores and we'll be quite close to the final node spec. (edited)

2024-11-21 12:34 gav: close enough, i expect, for effective modelling and experimentation. (edited)

2024-11-21 12:34 xlchen: is it possible to expose some heat sink somehow for it to be used as a real toaster

2024-11-21 12:35 gav: i'll ask :)

2024-11-21 14:44 danicuki: > <@gav:polkadot.io> Yes - at the Polkadot Palace in Cascais What is the Polkadot Palace? Is it a space open to community? Can we make JAM0 there in March/2025?

2024-11-21 14:50 gav: Yes, JAM0 gatherings can probably take place there in due course. But it's currently under heavy redevelopment and is unlikely be usable before 2026.

2024-11-21 14:54 danicuki: Maybe we can make JAM0/March in a nearby hotel, close to Cascais, and have a tour to the datacenter. Would this be possible? I am already planning next JAM0, this is so exciting!

2024-11-21 15:44 subotic: > <@danicuki:matrix.org> Maybe we can make JAM0/March in a nearby hotel, close to Cascais, and have a tour to the datacenter. Would this be possible? I am already planning next JAM0, this is so exciting! I would love that 👍

2024-11-21 17:00 gav: Definitely

2024-11-21 17:03 boymaas: Amazing! 😎 It’s an honour to be part of this process so unique to our times. 🚀✨ 

2024-11-21 17:08 dakkk: I made this tool in order to track graypaper equations inside jampy between different graypaper releases. If someone needs it, it is public: https://github.com/dakk/jambiato (edited)

2024-11-21 17:08 dakkk: image.png

2024-11-21 17:52 danicuki: > <@gav:polkadot.io> Definitely Please share the dates you will be in town in March/25 so we setup things to have you with us during JAM0.

2024-11-22 04:43 rithy: I came late in the group. Which country is JAM0 planning to host?

2024-11-22 05:54 danicuki: > <@rithy:matrix.org> I came late in the group. Which country is JAM0 planning to host? Next JAM0 will be in Portugal / March 2025

2024-11-22 06:17 rithy: Thank you for the information 

2024-11-22 12:37 gav:

2024-11-22 13:05 danicuki: > <@rustybot:matrix.org> Right. I'll move it to the state I propose something similar for "output" in test vectors. Why do we have this "output"? Why don't simple put every side effect of STF into post_state branch? For exemple, here:

consume_authorization_once-1.json

            "reported": [
                {
                    "work_package_hash": "0x63c03371b9dad9f1c60473ec0326c970984e9c90c0b5ed90eba6ada471ba4d86",
                    "segment_tree_root": "0x5c9dca800c145685f052ce1ba343e2b66b4c333ee4c4ca3b29cb347b1009cb83"
                }
            ],
            "reporters": [
                "0x22351e22105a19aabb42589162ad7f1ea0df1c25cebf0e4a9fcd261301274862",
                "0x5c7f34a4bd4f2d04076a8c6f9060a0c8d2c6bdd082ceb3eda7df381cb260faff",
                "0x837ce344bc9defceb0d7de7e9e9925096768b7adb4dad932e532eb6551e0ea02"
            ]

would it be better to put these "output" into ρ on post_state?

2024-11-22 13:08 danicuki: Generally speaking, test vectors should have only: "input" => contain all extrinsic and header information "pre_state" => all info related to previous state "post_state" => expected post_state will all relevant components change "output" => only "error / error name" or "ok"

2024-11-22 15:23 ycc3741: Hi, I wonder wether we should sort the hash value by "Lexicographical Order".

2024-11-22 20:58 celadari: Hey guys, a few questions on the work report test vectors: - in the state part of the .json ("pre_state" or "post_state") we have fields bytes and items => what do they represent in the GP ? If I go to the chapter 9,service accounts, I see s, p, l, c, b, g, m but I don't see what fields they could represent. I guess I would have to parse/decode bytes and items to get p, l, s but I don't see how. I feel there is something I'm not seeing. Also there are 6 fields in the .json but the GP has 7 elements. - I put here a question I put in a comment (just to centralize my questions): in the "output" field we have the field "reported" => what variable in the GP does it represent ? (edited)

2024-11-22 21:11 jaymansfield: > <@celadari:matrix.org> Hey guys, a few questions on the work report test vectors: > > - in the state part of the .json ("pre_state" or "post_state") we have fields bytes and items => what do they represent in the GP ? If I go to the chapter 9,service accounts, I see s, p, l, c, b, g, m but I don't see what fields they could represent. I guess I would have to parse/decode bytes and items to get p, l, s but I don't see how. I feel there is something I'm not seeing. Also there are 6 fields in the .json but the GP has 7 elements. > - I put here a question I put in a comment (just to centralize my questions): in the "output" field we have the field "reported" => what variable in the GP does it represent ? There is a definition of items/bytes at the start of section 9.3

2024-11-22 21:37 rick: Will the Safrole test vectors be updated to match the codec? Looks like it's just EpochMarks adding tickets-entropy, so nbd but it did trip up my tests.

2024-11-23 17:31 danicuki: I have a doubt about work report test vectors, on services list. What are "bytes" and "items" fields are? Should'nt s: Storage dictionary (D⟨H → Y⟩) p: Preimage lookup dictionary (D⟨H → Y⟩) and l: Preimage lookup dictionary be provided as well?

2024-11-23 17:39 danicuki: Also on the same tests, I don't see slot or next_validators provided. Is it correct? Because when slot is not provided in previous state, there is a key rotation and since next_validators are also not provided, we come to a situation with empty validators set

2024-11-23 17:43 jaymansfield: > <@danicuki:matrix.org> I have a doubt about work report test vectors, on services list. What are "bytes" and "items" fields are? Should'nt s: Storage dictionary (D⟨H → Y⟩) p: Preimage lookup dictionary (D⟨H → Y⟩) and l: Preimage lookup dictionary be provided as well? > There is a definition of items/bytes at the start of section 9.3

2024-11-23 17:44 jaymansfield: Slot # is provided in the input for the report vectors (it’s right above entropy) (edited)

2024-11-23 17:45 danicuki: > <@jaymansfield:matrix.org> Slot # is provided in the input for the report vectors (it’s right above entropy) I assume slot in the input is the header slot. What about pre_state slot?

2024-11-23 17:47 jaymansfield: > <@danicuki:matrix.org> I assume slot in the input is the header slot. What about pre_state slot? I think for pre state slot we just used the input slot subtracted by one

2024-11-23 17:48 danicuki: Why isn't this explicit in the test vectors? How could I guess this? (edited)

2024-11-23 17:56 jaymansfield: > <@danicuki:matrix.org> Why isn't this explicit in the test vectors? How could I guess this? It might not really matter what value you use. The tests only really cover validating guarantees and updating state afterwards. I don’t think we did anything special for epoch updates in relation to it.

2024-11-23 20:22 oliver.tale-yazdi: At JAM0 we talked briefly about how to construct the Dev account secret keys, I followed the approach of the test vectors now and updated the accounts here https://docs.jamcha.in/basics/dev-accounts (edited)

2024-11-24 14:48 gav: signal-2024-11-22-170556_002.jpeg

2024-11-24 14:48 gav: signal-2024-11-22-171645_002.jpeg

2024-11-24 14:48 gav: signal-2024-11-22-174613_002.jpeg

2024-11-24 14:48 gav: signal-2024-11-22-174723_002.jpeg

2024-11-24 14:48 gav: A few more pics from the building of the Toaster

2024-11-24 14:49 gav: The first batch of racks has arrived. One of the enclosures is finished. And there's the copper power cable (it's very thick) and conduit. That's for the ~100kW power draw. Thankfully the Palace has a ~300kW power line so we're good for now. (edited)

2024-11-24 14:55 boymaas: Racks in, power set, lots of movement!—this setup is getting ready to run some rock-solid, efficient Zig code. ⚡🚀😎

2024-11-24 15:36 emielsebastiaan: 1.21 Gigawatts Great Scott ⚡️

2024-11-24 17:45 prematurata: > <@boymaas:matrix.org> Racks in, power set, lots of movement!—this setup is getting ready to run some rock-solid, efficient Zig code. ⚡🚀😎 Or some interpreted code like typescript :)

2024-11-24 23:57 xlchen: I have published a docker image for Boka so that it can be used to test networking compatibility https://hackmd.io/HsBZpvxQSXOBMSu0Z3ciFA?view cc Oliver Tale-Yazdi - (back on 25th)

2024-11-25 03:11 sourabhniyogi: Can someone suggest a better name for this ambiguous JSON attribute tree_root https://github.com/w3f/jamtestvectors/blob/90fcf9020fa269a3aecc23feac0d09e7fee9123b/codec/data/work\_item.json#L8 like segment_root_or_work_package_hash before we fall in love with tree_root ... Origin of the concept is https://graypaper.fluffylabs.dev/#/911af30/19e30019e300 (edited)

2024-11-25 09:58 dakkk: davxy: regarding codec tests, is it correct to assume that data is valid? for instance, in the block test (https://github.com/w3f/jamtestvectors/blob/master/codec/data/block.json), should extrinsic_hash correspond to the computed hash of the given extrinsic?

2024-11-25 10:20 tomusdrw: AFAIR it was mentioned earlier that the codec test data is not valid semantically. It should be decoded successfully, but it doesn't necessarily make any sense in JAM context. (edited)

2024-11-25 10:21 dakkk: > <@tomusdrw:matrix.org> AFAIR it was mentioned earlier that the codec test data is not valid semantically. It should be decoded successfully, but it doesn't necessarily make any sense in JAM context. ok my bad (y) thank you for re-clarifying it

2024-11-25 10:28 rustybot: @[dakkk | JamPy] https://github.com/w3f/jamtestvectors/tree/master/codec#semantic-correctness

2024-11-25 10:29 dakkk: > <@rustybot:matrix.org> @[dakkk | JamPy] https://github.com/w3f/jamtestvectors/tree/master/codec#semantic-correctness yep, understood

2024-11-25 11:48 tomusdrw: I believe the W_X in export host call definition: https://graypaper.fluffylabs.dev/#/911af30/33ae0033b100 should rather be W_M (i.e. the maximal number of imported/exported items). The former doesn't seem to be defined anywhere. (edited)

2024-11-25 11:57 gav: > <@tomusdrw:matrix.org> I believe the W_X in export host call definition: https://graypaper.fluffylabs.dev/#/911af30/33ae0033b100 should rather be W_M (i.e. the maximal number of imported/exported items). The former doesn't seem to be defined anywhere. https://github.com/gavofyork/graypaper/pull/150/commits/968dc94aec3e9cada4b177b44ef7320a16ecc434

2024-11-25 11:57 gav: Will be in 0.5.1

2024-11-25 19:12 piotrzwolinski:

2024-11-25 19:14 piotrzwolinski: Hello JAM Community! Over the past few weeks, the FluffyLabs team has been finalizing the first official release of the PVM Debugger (https://pvm.fluffylabs.dev). Alongside this, we’re preparing a proposal for retroactive funding from the Polkadot Treasury to support the development of this tool. We also plan to continue expanding its features in the near future. We’ve started a discussion about the planned Treasury proposal on Polkassembly: https://polkadot.polkassembly.io/post/2578 and would greatly appreciate your feedback. Please don’t hesitate to ask any questions or share your thoughts!

2024-11-26 18:38 gav: Initial version of Parity JAM SDK is out, along with the Bootstrap Service: Parity JAM SDK

2024-11-26 18:39 gav: It's not much right now and I've only tested it locally. But (if it works) it should be enough for anyone to build the Bootstrap service PVM code blob and (vaguely) see its Work Item ABI. (edited)

2024-11-26 18:39 gav: Rust coders should also be able to code and build their own services and authorizers.

2024-11-26 18:40 gav: The docs are a WiP, and I've thus far focussed only on basic Bootstrap service description. They should be improved in the coming days.

2024-11-26 18:41 gav: NOTE: There's no Github repo open yet, since we have a monorepo together with our node impl. So the code is currently available on crates.io only.

jam-pvm-build ./jam-bootstrap-service
jam-pvm-build ./jam-null-authorizer

# cargo run -p polkatool disassemble --show-raw-bytes ~/go/src/github.com/jam-duna/jamtestnet/services/parity/jam-bootstrap-service.pvm
warning: /root/go/src/github.com/colorfulnotion/polkavm/Cargo.toml: unused manifest key: workspace.lints.rust.unexpected_cfgs.check-cfg
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
     Running `target/debug/polkatool disassemble --show-raw-bytes ~/jam-duna/jamtestnet/services/parity/jam-bootstrap-service.pvm`
ERROR: failed to parse "/root/go/src/github.com/jam-duna/jamtestnet/services/parity/jam-bootstrap-service.pvm": failed to parse program blob: blob doesn't start with the expected magic bytes

2024-11-27 04:28 xlchen: I plan to focus on the RPC part for Boka next and that includes the ability to generate RPC docs. Oliver Tale-Yazdi suggested to use https://github.com/ethereum/execution-apis / https://ethereum.github.io/execution-apis/api-documentation/ as the playground / documents for RPCs. Let me know what do you think

2024-11-27 18:42 danicuki:

2024-11-27 18:45 danicuki: Report test vectors seem to be using max gas GA = 1,000,000 - but GP sets to 100,000 - any particular reason for using a different number? davxy (edited)

2024-11-27 18:45 danicuki: Formula (11.29) ∀w∈w∶ ∑(rg)≤GA

2024-11-27 18:54 davxy: See discussion with @dakkk:matrix.org here https://github.com/w3f/jamtestvectors/pull/20

2024-11-27 18:55 davxy: I need to fix it

2024-11-27 19:06 danicuki: > <@davxy:matrix.org> See discussion with @dakkk:matrix.org here https://github.com/w3f/jamtestvectors/pull/20 thanks for sharing. Solved another doubt I had about "duplicates" / "out of order" cores

2024-11-27 19:21 danicuki: It would be REALLY REALLY helpful if vectors README had GP Formula Numbers / links for each condition being tested. e.g.:

- [not_authorized](./tiny/not_authorized-1.json) 🔴
  - Target core without any authorizer.

I can't find out from this description what this test is all about. Sometimes the phrasing explanation is not accurate. A simple link to a formula would solve any doubt (edited)

2024-11-27 19:27 gav: JAM SDK API docs are slowly improving https://docs.rs/jam-pvm-common/0.1.5/jam_pvm_common/

2024-11-27 19:34 danicuki: davxy: what value are you using for rotation period (R) in test vectors?

2024-11-27 22:32 danicuki: An interesting article that brings even more excitement to our JAM builders community: https://user.fm/files/v2-261b7914c204931fbf213d7d35307264/worldcomputer.pdf

2024-11-28 09:26 gav: > <@danicuki:matrix.org> An interesting article that brings even more excitement to our JAM builders community: https://user.fm/files/v2-261b7914c204931fbf213d7d35307264/worldcomputer.pdf TBH I don't think it'll be a single "World Computer" any more.

2024-11-28 09:26 gav: At least as long as it is not totally free.

2024-11-28 09:27 gav: More likely, we'll see several weakly-compatible ("bridged") World Computer networks.

2024-11-28 09:27 gav: Kinda similar to the cloud services at present.

2024-11-28 09:28 gav: Each with their own tradeoffs in terms of reliability, scale, cohesion and peripheral services.

2024-11-28 09:28 gav: JAM as a protocol would be the architecture of the (first wave, anyway) of machines on the Polkadot "World Computer Network".

2024-11-28 09:30 gav: But the idea is for multiple JAMs to be placed in a sort of grid network, similar in approach to the Omniledger protocol. This seems like the only sensible option at present to scale more-or-less indefinitely. (edited)

2024-11-28 09:40 sourabhniyogi: As an old guy (so old I have seen a "Cray" and "Thinking Machines"), I really adore the shift to the words "trustless supercomputer" more than the "World Computer", where it matches 1 core = World Computer, N cores = Supercomputer. In a completely ordinary way, I think the term "SUPERcomputer" works on people well because of the superhero vs hero distinction that loosely maps onto your favorite word "Core", which ALSO works for everyday PC computer buyers. Why not be consistent with this new "supercomputer" term for marketing reasons? (edited)

2024-11-28 09:44 sourabhniyogi: Polkadot JAM Chain is the world's first trustless SUPERCOMPUTER because ____ (a) it has more than one core! (b) it is actually semi-coherent! You get to decide the truthiness of (a)+(b) =)

2024-11-28 09:51 gav: I remember back in the old days (~y2k), pre-cloud, there was the concept of the "grid", basically a network of supercomputers with streamlined interconnects and access.

2024-11-28 09:53 gav: It seems unlikely to me that any fully-coherent system is going to scale indefinitely and economics being what they are, it seems unlikely that strict coherence will be paid for when it's not needed.

2024-11-28 09:54 gav: This all points to a multi network stable-state.

2024-11-28 09:55 gav: But probably with a range of interoperability/cohesion across the various (super)computers which make it up. (edited)

2024-11-28 09:55 gav: I don't really see crappy (trustful) bridges in this, except perhaps to connect low-quality low-resilience not-really-web3 networks.

2024-11-28 10:05 gav: But then probably we'll see a spectrum of connectivity; trustless light-client bridges, L1-L2 and other one-way full-node (ala Eth L2s and parachains), dual-way full-node, shared validators (ala Omnipool), weakly-coherent shared DA (ala JAM) and fully-coherent synchronous shared state (ala Eth L1).

2024-11-28 10:07 gav: We're already seeing a bit of this, but it's very much at the poles so far (incoherent or fully synch).

2024-11-28 10:11 gav: For the Polkadot Cloud Grid, we could almost certainly make a matrix of JAMs with a shared validator pool and light-client bridge interconnects (at the cost of a ~35% drop in cost of attack). This would be a basic Supercomputer Network. We can probably do better though, with a shared DA and maybe some built-in service migration or even Work Report and Transfer forwarding, so that Refine is spread around the network but Accumulation/OnTransfer still happens synch on a single "home JAM" of the service. (edited)

2024-11-28 10:12 gav: This would provide totally unprecedented levels of scale; an e.g. a 10x10x10 grid would give 1000x on the total compute throughput of JAM - so like ~1 quadrillion EVM-equiv gas/sec (edited)

2024-11-28 10:16 gav: There's still some stuff to be worked out, like preimage fetching/sharing across JAMs (possibly a simple cache model would do, or perhaps also an explicit fetch) and DA handoff as the partitioning changes (possibly just ephemeral connections across JAM validator partitions for segment-fetching). (edited)

2024-11-28 10:17 gav: But overall I see it as a pretty interesting next step once JAM is solid.

2024-11-28 10:19 gav: We should also keep our eyes on the progression of ZK stuff too. I think (and this is almost exclusively informed by the W3F research team) that it's too early to tell yet whether it'll become economically viable compared to ELVES, but if it does eventually come down in price enough to be a sensible option when we could potentially see a backward-compatible JAM-ZK on the cards in a few years.

2024-11-28 10:21 danicuki: davxy: are safrole test vectors already updated to GP 0.5? I see some of our tests failing after we upgraded our code to 0.5 specification

2024-11-28 10:22 emielsebastiaan: This made my day thx for sharing these insights once in a while.

2024-11-28 10:24 davxy: > <@danicuki:matrix.org> davxy: are safrole test vectors already updated to GP 0.5? I see some of our tests failing after we upgraded our code to 0.5 specification Which ones? And what is the actual failed condition? They should be aligned to 0.5

2024-11-28 12:07 gav: Sure if you want to set it up

2024-11-28 12:32 davxy: Was 4 for both tiny and full. I've just fixed it to be 10 for full. The values are NOW explicitly reported in the README (edited)

2024-11-28 19:13 davxy: https://github.com/w3f/jamtestvectors/pull/8/commits/df18b318433fda0cf0dd2b4aac29abce09f0b15a

2024-11-28 20:43 davxy: Could this be the breaking change? https://github.com/w3f/jamtestvectors/pull/8/commits/aeb976ff3eed8afed4dda53286f7cb2af5b81ebd Specifically, post-offenders have been moved to the "state". Please refer to the ASN.1 schema for further details. I followed the recommendation to include in the "input" struct only the elements found within the block (I.e. header items and extrinsics). When a pre-execution state item is "posterior" it is explicitly specified in the ASN.1 schema. Always keep an eye on the schema. (edited)

2024-11-28 20:50 davxy: ^ posterior with respect to some other subsystem STF (in this case, it is the "disputes" subsystem that advances the offenders' list). Thus, it is assumed that the subsystem has already been executed to obtain psi_o'.

2024-11-28 23:26 danicuki: About the work report vectors: why the post_state doesn't include the core report component - with all its fields? Shouldn't be better than using an "output" in the json? @davxy

2024-11-29 06:38 xlchen: Trying to use OpenRPC tools for JAM RPC spec work. It is kind of working: https://playground.open-rpc.org/?schemaUrl=https://raw.githubusercontent.com/open-web3-stack/jam-rpc-spec/refs/heads/master/openrpc.json The spec defs is generated by our Swift code from our RPC implementation so those methods are what we have implemented currently, not what I wanted to propose

2024-11-29 06:40 xlchen: I created a repo here but not sure if it should just be part of some other repos https://github.com/open-web3-stack/jam-rpc-spec?tab=readme-ov-file

2024-11-29 06:43 xlchen: I plan to create the actual spec proposal (in Swift code) and update the openrpc spec file next week

2024-11-29 06:46 xlchen: open-rpc have client and server generator which I didn't try, but the docs generated don't work for me https://github.com/open-rpc/generator

2024-11-29 14:32 nathanccxv: I recently finished reading Jam's GrayPaper and am preparing to develop its implementation. Is it too late for that? Is there any time limit? gav

2024-11-29 15:09 gav: There’s no time limit, though of course the prize pool is limited and it’s FCFS :)

2024-11-29 15:26 nathanccxv: Thank you for your clarification. According to the 'Gav's Unofficial JAM Prize Notes' document, up to five implementations per language set can receive rewards. 2500k/(100k*5 milestones) = 5. Is this understanding correct?

2024-11-29 16:30 dakkk: > <@nathanccxv:matrix.org> I recently finished reading Jam's GrayPaper and am preparing to develop its implementation. Is it too late for that? Is there any time limit? gav if may I ask, which language are you going to use?

2024-11-29 16:32 gav: Well, this assumes each of those 5 implementations get to M5 before any others get to M1. (edited)

2024-11-29 21:50 tomusdrw: > <@gav:polkadot.io> Well, this assumes each of those 5 implementations get to M5 before any others get to M1. Hmm, I guess that creates a risk of having no incentives to go up to M5 for some of the "heavily occupied" language sets depending on the timing of when it will be possible to submit implementations for acceptance, doesn't it? But AFAIR I saw in the repo, that there is not going to be milestone deliveries before GP 1.0, right?

2024-11-29 22:27 gav: Indeed

2024-11-29 22:33 sourabhniyogi: I'm putting together a "importblocks" fuzz tester and would like to know which state elements would be changed by blocks in "M1 Import blocks" milestone from https://graypaper.fluffylabs.dev/#/293bf5a/348e00348e00 in different testing modes to match the goals of M1 vs M2 vs M3 as well as which extrinsics. I think we should have a super precise answer for M1 Import Blocks now? My guess is: - "M1 Import blocks" will have blocks that change the state of: - C(4), C(6), C(7), C(8), C(9), C(10), C(11), C(13), C(15) - all service storage (δ, a\_s, a\_p, a\_l) - extrinsics E\_{T,G,A,P} ... implying these will happen later: - M2/M3: C(1), C(2), C(3), C(5), C(12), C(14), E\_D Can someone clarify what M1 Import Blocks state changes should cover? This has implications as to what teams should focus on finishing in what order. I remember earlier comments that M1 doesn't even need PVM implementation, which is at odds with the above guess. This is important for the design+implementation of a fuzz tester that only generates blocks suitable for each milestone / mode, with a parameter like "mode" that is like M1: "fallback" (no extrinsics), "safrole" (only ticket extrinsics), "assurances" (all but dispute extrinsics and no prereqs) M2: "orderedaccumulation" (adjusting C(14), including prereqs), "authorization" (adjusting C(1)+C(2)), "recenthistory" (C(3)), "blessed" (adjusting C(5), C(12)), "basichostfunctions" (most common host functions) M3: "finalization" (everything in C(1)-C(15) except disputes), "disputes" (everything, including dispute extrinsics), "conformance" (every single host function) (edited)

2024-11-29 23:05 charliewinston14: Anyone else having difficulty verifying the beefy_root's in the report vectors? Just wondering if it's my issue or if its an issue with the vectors themselves?

Hash.keccak_256(Codec.Encoder.encode_mmr(y.accumulated_result_mmr))

2024-11-30 06:07 ksc85pwpj5: davxy: Excuse me, may I kindly ask if there are currently any test vectors available for shuffle, or if anyone else has any results related to shuffle?

2024-11-30 06:41 dakkk:

2024-11-30 06:42 dakkk:

2024-11-30 07:04 rustybot:

2024-11-30 07:04 davxy: > <@sourabhniyogi:matrix.org> I'm putting together a "importblocks" fuzz tester and would like to know which state elements would be changed by blocks in "M1 Import blocks" milestone from > https://graypaper.fluffylabs.dev/#/293bf5a/348e00348e00 > in different testing modes to match the goals of M1 vs M2 vs M3 as well as which extrinsics. I think we should have a super precise answer for M1 Import Blocks now? > > My guess is: > > - "M1 Import blocks" will have blocks that change the state of: > > - C(4), C(6), C(7), C(8), C(9), C(10), C(11), C(13), C(15) > - all service storage (δ, a\_s, a\_p, a\_l) > - extrinsics E\_{T,G,A,P} > > ... implying these will happen later: > > - M2/M3: C(1), C(2), C(3), C(5), C(12), C(14), E\_D > > Can someone clarify what M1 Import Blocks state changes should cover? > > This has implications as to what teams should focus on finishing in what order. I remember earlier comments that M1 doesn't even need PVM implementation, which is at odds with the above guess. > > This is important for the design+implementation of a fuzz tester that only generates blocks suitable for each milestone / mode, with a parameter like "mode" that is like > M1: "fallback" (no extrinsics), "safrole" (only ticket extrinsics), "assurances" (all but dispute extrinsics and no prereqs) > M2: "orderedaccumulation" (adjusting C(14), including prereqs), "authorization" (adjusting C(1)+C(2)), "recenthistory" (C(3)), "blessed" (adjusting C(5), C(12)), "basichostfunctions" (most common host functions) > M3: "finalization" (everything in C(1)-C(15) except disputes), "disputes" (everything, including dispute extrinsics), "conformance" (every single host function) https://github.com/w3f/jamtestvectors/issues/21

2024-11-30 07:55 davxy: It seems your tool aligns somewhat with the conformance testing tool for M1. I encourage the sharing of relevant ideas and discussions within the GitHub issue, as people (me included) may occasionally miss messages in the chat

2024-11-30 08:07 prematurata: > <@davxy:matrix.org> https://github.com/w3f/jamtestvectors/issues/21 hey davxy thanks for this. I previously used your polkajam-vectors branch but i noticed that compared to the upstream repository gamma_z is computed differently (my findings were that a different version of ark repo was used.) caan you share some insights on which of the 2 provide the "correct" gamma_z computation?

2024-11-30 10:51 davxy: I checked the value of gamma_z and is the same in both the upstream PR and my fork. Obviously if for upstream you mean the master branch, yes. This is different. At this point master is quite outdated. I suggest you to work with the changes in the PR or using my fork, that I try to keep updated WRT the open PRs we have upstream. I report the **current** values here: ### Tiny "gamma\_z": "0xa949a60ad754d683d398a0fb674a9bbe525ca26b0b0b9c8d79f210291b40d286d9886a9747a4587d497f2700baee229ca72c54ad652e03e74f35f075d0189a40d41e5ee65703beb5d7ae8394da07aecf9056b98c61156714fd1d9982367bee2992e630ae2b14e758ab0960e372172203f4c9a41777dadd529971d7ab9d23ab29fe0e9c85ec450505dde7f5ac038274cf", ### Full "gamma\_z": "0x96fcd017ddd980b1847422ee6d9c7619e71d1a17acfbb9c572feec917cf3c12f5a87d33f249b5ecf2639859bf3eb2795841d5317144feab07f6e7d0add7a8b3591b9c913ea56198c53a0894f040d7f1392c790755b97a6167e0ca9978421e6b596c1b168e2dcc743f9eadda76c041db42d39f27a58418f88c0ea67656a224934e12b5dfc8f0f460a95c2d467fa41907b", (edited)

2024-11-30 10:51 davxy: You can check for the ark-ec-vrfs revision currently used here: https://github.com/davxy/jam-test-vectors/tree/polkajam-vectors/safrole#%EF%B8%8F-warning--%EF%B8%8F

2024-11-30 10:58 davxy: Following suggestions from some other team members, the tiny configuration now uses a maximum tickets per validator of 3 (compared to 2 for the full configuration). This adjustment ensures that epoch slots can be filled with tickets from a supermajority of honest nodes (5 out of 6 validators). This change aligns the vectors to the proposed testing values outlined here: https://docs.jamcha.in/basics/chain-spec/Tiny.

2024-11-30 11:47 sourabhniyogi: > <@davxy:matrix.org> It seems your tool aligns somewhat with the conformance testing tool for M1. I encourage the sharing of relevant ideas and discussions within the GitHub issue, as people (me included) may occasionally miss messages in the chat https://github.com/w3f/jamtestvectors/issues/21#issuecomment-2508934671

2024-11-30 12:28 tomusdrw: > <@gav:polkadot.io> JAM SDK API docs are slowly improving https://docs.rs/jam-pvm-common/0.1.5/jam_pvm_common/ I'm playing around with the SDK and have some .pvm files compiled. It seems to me that there is one extra byte between "read-only data" o and E_4(codeLength). Normally there should be w there, but length of w is set to 0. Am I missing something, or might this be a bug?

2024-11-30 13:47 gav: We won’t

2024-11-30 14:59 gav: Would need to check with @jan:parity.io

2024-11-30 15:50 gav: Yeah it’s possible @jan didn’t yet update it.

2024-11-30 16:02 subotic: I have an open PR with the updated opcodes. Jan wants to wait with merging.

2024-11-30 16:04 subotic: > <@subotic:matrix.org> I have an open PR with the updated opcodes. Jan wants to wait with merging. The reason is that there are smart contracts on Westend, which would otherwise break.

2024-11-30 16:14 subotic: > <@subotic:matrix.org> The reason is that there are smart contracts on Westend, which would otherwise break. Furthermore, there are additional opcodes coming with the ZBB extension, and there might be another round of opcode reshuffling. This is what I have understood anyway. Jan is the authority, though.

2024-11-30 16:50 tomusdrw: Yeah, no worries. I was just trying to figure out if it's possible to import the service code into PVM disassembler. I think I do understand where we are at, so it's fine and I can wait for full GP-0.5.0 migration.

2024-12-01 11:55 jan: So we've recently switched to a 64-bit PVM in the GP. I haven't got the updated the test vectors ready yet (I will have them ready soon-ish), but in case some of you are already looking to update your PVM implementations to 64-bit here are some tips to make this process easier: - The major change introduced with the 64-bit PVM is that the registers are now 64-bit, and in case of the most of the instructions their behavior stays essentially exactly the same, except they operate on 64-bit values instead of on 32-bit values now. - The instruction encoding is unchanged. Those instructions which previously took at most a 32-bit immediate value (like e.g. and_imm) now still can have at most a 32-bit physical immediate, however the immediate value is now sign extended to full 64-bit before being used. - There's only a single instruction which takes a 64-bit immediate that is actually physically encoded in the code stream as 64-bit (load_imm_64) - There are a couple of instructions where there exists both a 64-bit variant and a 32-bit variant. (e.g. add_32 and add_64) The _32 variants work like the old 32-bit instruction from 0.4 - they ignore the upper 32-bits of the registers and use only the lower bits, but their result is always sign extended to full 64-bits when written to the destination register. - The address space is still 32-bit and memory accesses ignore the upper 32-bits of registers. If any of you have trouble updating your PVMs or have any questions regarding PVM then, as always, feel free to ping me and I'll be happy to help.

2024-12-01 17:19 danicuki: What is the difference between tickets_per_validator: 3 and max_tickets_per_extrinsic: 3 in tiny vectors specification? Aren't the the same thing? https://docs.jamcha.in/basics/chain-spec/tiny

2024-12-01 18:58 sourabhniyogi: I tabulated the 0.4.5 opcode => 0.5.0 opcode updates here https://docs.google.com/spreadsheets/d/1R7syeL7GYq4KH2B3Zh03v3CAFnK1iNNF0J4c2r-vKWw/edit?gid=1743195954#gid=1743195954 If you think the sheet is wrong, just update the sheet and point out what you changed.

2024-12-01 19:18 jaymansfield: > <@sourabhniyogi:matrix.org> I tabulated the 0.4.5 opcode => 0.5.0 opcode updates here > https://docs.google.com/spreadsheets/d/1R7syeL7GYq4KH2B3Zh03v3CAFnK1iNNF0J4c2r-vKWw/edit?gid=1743195954#gid=1743195954 > > If you think the sheet is wrong, just update the sheet and point out what you changed. Thank you for this!

2024-12-02 12:02 danicuki: davxy: How do you setup the initial value for authorizer queue (φ) for work report vectors? They are not supposed to be empty. Formula 8.2 adds φ[c][Ht] to authorizer pool, but in vectors they are not being added.

2024-12-02 12:13 danicuki: What are "offenders" state field in the work report vectors? Are they part of the psi state field? If so, please, use the same convention you used on disputes vectors. Please, be consistent with test vectors, regarding to json field names, use the same convention on all vectors, otherwise we get crazy ✌️ (edited)

2024-12-02 12:36 rustybot: The asn.1 syntax is clear about it

2024-12-02 12:37 davxy: https://github.com/davxy/jam-test-vectors/blob/work-reports/reports/reports.asn

2024-12-02 12:48 danicuki: Thanks for the clarification. the .asn helps a lot. My suggestion is more about using the same names / structure for the same field in different test vectors. e.g.: why not use the same naming convention that you were already using in safrole and disputes vectors (psi, rho, tau, kappa, etc)?

2024-12-02 12:55 davxy: I believe that reading the ASN.1 syntax file is more than enough to eliminate any ambiguity. Regarding the state, I’m open to renaming it if it helps. However, at the time of writing, I felt that offenders was a clearer choice compared to psi_o_prime. :-) (edited)

2024-12-02 13:34 rick: Hard agree +1 to offenders over psi_o_prime, you've already got defs in the GP which make that link to the asn spec name.

2024-12-02 14:04 danicuki: > <@davxy:matrix.org> I believe that reading the ASN.1 syntax file is more than enough to eliminate any ambiguity. Regarding the state, I’m open to renaming it if it helps. However, at the time of writing, I felt that offenders was a clearer choice compared to psi_o_double_dagger. :-) > > My suggestions is to use offenders, but ender psi branch: Psi: { offenders: [...] other_psi_fields... } This is how it is in disputes vectors, which for me makes much more sense. Offenders under the main state branch seems odd, as state doesn't have an "offenders" component.

2024-12-02 14:30 davxy: I understand your perspective. However, the state in question is not the "main" state. Only the relevant portions of the state required to advance the STF of a particular subsystem are included. In cases of "disputes" vectors, the entire ψ is necessary, while "reports" STF requires only ψₒ. I don't think that including items that are not strictly needed is beneficial, which is why for example, I have not included other state items here. I recognize that this approach is based on personal preference. However, since there is no formal specification for how to write vectors, I believe it is more efficient to exclude elements not required by the subsystem's STF. If anyone wishes to draft a mini-specification for writing vectors, I encourage you to open an issue on GitHub to address these considerations. Of course, I will eventually adapt my approach to the issue outcome. (edited)

2024-12-02 15:41 sourabhniyogi: Strongly prefer long Developer-friendly English attribute names in JSON over short greek letters from GP to reduce cognitive load. Its best to have a single benevolent dictator have good consistent taste than have any inconsistency in this. The style is roughly in place already, lets not change it! For STF test vectors: - raw k,v values in JSON with a little bit of metadata for service storage sufficient to derive the key and know its type will be best. - having a comprehensive dump of raw k,v values for the "pre\_state" is necessary, ideally coupled with this metadata; - for the post\_state, its a debate as to whether to have a comprehensive dump of k,v state along with this metadata. Its easier to read/write but a tad harder to spot the differences without writing a little diff code. - Having the post state root is desirable but not strictly necessary. - Having a JAM codec version of an STF test vector might be useful or unnecessary. But, in general, the W3F test vectors being in both JSON + JAM Codec shuts down unnecessary debates effectively. In all cases, having W3F be the benevolent dictator is very useful so we don't have engineers bickering over formats. (edited)

2024-12-02 17:13 oliver.tale-yazdi: > <@davxy:matrix.org> I believe that reading the ASN.1 syntax file is more than enough to eliminate any ambiguity. Regarding the state, I’m open to renaming it if it helps. However, at the time of writing, I felt that offenders was a clearer choice compared to psi_o_prime. :-) IMHO any naming is fine. I only write these things once into a translation map and never think about them again.

2024-12-02 20:48 danicuki: I had a doubt about the assurance bitfield interpretation in vectors: I see in vectors we have values from 0x01 to 0x03, which would be: \[0, 1\], \[1, 0\] and \[1, 1\] My question is: what is the correct position for cores? Do we use the least significant bit order or the most significant? In other words, if bitstring is 0x01 for example, core\[0\] is 1 and core\[1\] is 0? Or the opposite? Having some issues with af[c] with the presented bitfields (edited)

2024-12-02 21:01 jaymansfield: We have these vectors passing now in our implementation. For bitfield 0x01, the vectors seem to expect core[0]=1 core[1]=0, and 0x02 to be core[0]=0 core[1]=1. Had the opposite originally and had to reverse the binary we generated for it to pass. (edited)

2024-12-03 03:22 xlchen: Here is my RPC proposal draft: https://github.com/open-web3-stack/jam-rpc-spec pretty much copying the Substrate RPC with some minor tweaks

2024-12-03 04:28 amritj: In graypaper here https://graypaper.fluffylabs.dev/#/911af30/19e30019e600 It is mentioned that import spec could contain both segment root and work package hash and to distinguish between them the work package hash value are tagged But here according to jam official codec it is just a hash: https://github.com/w3f/jamtestvectors/blob/90fcf9020fa269a3aecc23feac0d09e7fee9123b/jam-types-asn/jam-types.asn#L121 So, how do we identify, if hash is tagged or not i.e. is it a segment root or work package hash? (edited)

2024-12-03 08:53 davxy: > <@amritj:matrix.org> In graypaper here https://graypaper.fluffylabs.dev/#/911af30/19e30019e600 > > It is mentioned that import spec could contain both segment root and work package hash and to distinguish between them the work package hash value are tagged > > > But here according to jam official codec it is just a hash: > > https://github.com/w3f/jamtestvectors/blob/90fcf9020fa269a3aecc23feac0d09e7fee9123b/jam-types-asn/jam-types.asn#L121 > > So, how do we identify, if hash is tagged or not i.e. is it a segment root or work package hash? Ty for reporting, I'll fix it.

2024-12-03 09:01 davxy: @room If you are using GitHub, please report any issues related to test vectors directly here. I may overlook or forget about some messages shared in the chat. This request does not pertain to trivial questions but rather to issues you are confident you have identified. Ty

2024-12-03 14:13 danicuki: I've been posting many questions related to test vectors here in this chat. Do you believe it is a good idea to have another room just for test vectors related questions, so we don't spam the main chat with these messages?

2024-12-03 14:50 danicuki: for no_assurances_with_stale_report-1 vector, how can the cores[0] report be nullified on the next state if there are no assurances in the block (thus no new report available)? I am referring to Formulas 11.15 and 11.16 - am I missing something?

2024-12-03 18:18 jam_man: Question about starting an implementation. Is there any guide(s) to follow, or is the idea that we use only the gray paper as a guideline for how to create the implementation?

2024-12-03 18:24 jam_man: More specifically. Is the idea that we read and interpret the gray paper and turn the core ideas into functional code. Specifically making sure we hit the milestones from the web3 rules page. (edited)

2024-12-03 18:52 danicuki: > <@jam_man:matrix.org> Question about starting an implementation. > > Is there any guide(s) to follow, or is the idea that we use only the gray paper as a guideline for how to create the implementation? We all started like this: read the paper, start writing code. You will get surprised about how much you can do. I've presented a workshop last month that might give you some ideas: https://www.youtube.com/watch?v=8UT2akmxu4I

2024-12-03 18:53 jam_man: > <@danicuki:matrix.org> We all started like this: read the paper, start writing code. You will get surprised about how much you can do. I've presented a workshop last month that might give you some ideas: https://www.youtube.com/watch?v=8UT2akmxu4I Awesome I will take a look at this. Thank you

2024-12-03 20:09 sourabhniyogi: I tabulated the W3F Test Vectors concerning "block validation" errors (safrole, reports, assurances, disputes) here: https://docs.google.com/spreadsheets/d/1yizEboIB28AP1fZXVfcm7kI5CtKd12TepsUcTDV1auo/edit?gid=2118274177#gid=2118274177 I assembled this to address a "importblocks" implementation: https://docs.jamcha.in/testing/import-blocks but this may help others develop systematic error treatment (in response to the above, and thus pass M1 Import Blocks). If you see fixes or want to add 0.5.x links to GP sections, just update the sheet! (edited)

2024-12-04 03:32 luke_jamixir: @davxy:matrix.org any plans to add to safrole vectors with proper header seal and vrf signatures? That would be very useful

2024-12-04 03:37 amritj: Question: The segments are erasure-coded and distributed to the validators. Each validator receives a single shard out of the 1,023 shards, determined by their validator index. Therefore, the shard index of the shard they receive equals to their validator index. According to the JAMNP protocol, to fetch data from the assurers, we need to provide each assurer with the shard index we require from them. The exported segments are expected to remain available for 28 days. During this period, it is assumed that more than 341 validators will remain consistent. However, there is still a significant likelihood that validator indexes may change during this time due to the addition or removal of validators. Given this, should we also maintain a record of the validator keys associated with each shard index? This would allow us to identify which validator holds a specific shard, regardless of changes in their validator index. Finally, is the assumption that more than 341 validators will remain unchanged for 28 days accurate? (edited)

2024-12-04 03:50 clearloop: May I ask if the example in https://github.com/davxy/bandersnatch-vrfs-spec is outdated? both the latest on main and @8c82722 could not be compiled (edited)

2024-12-04 04:23 luke_jamixir: > <@clearloop:matrix.org> May I ask if the example in https://github.com/davxy/bandersnatch-vrfs-spec is outdated? both the latest on main and @8c82722 could not be compiled Try to update the ark-ec-vrfs dependency to revision: e491b41

2024-12-04 04:37 luke_jamixir: Indeed, does not compile

2024-12-04 04:44 clearloop: seems caused by ring-proof is broken via the update of w3f/fflonk

2024-12-04 04:50 clearloop: Screenshot 2024-12-04 at 11.49.25.png

2024-12-04 04:50 clearloop: okay yesterday... in case anybody meet the same problem while playing around with it, you can add this patch to your local testing repo instead of updating the whole dependency chain

[patch."https://github.com/w3f/fflonk"]
fflonk = { git = "https://github.com/clearloop/fflonk", rev = "1e854f35e9a65d08b11a86291405cdc95baa0a35" }

(edited)

2024-12-04 07:42 davxy: mmm... I know. The problem is that these crypto backends are not yet published on crates.io. And stuff may break. I'm going to anchor the ring-proof we're using to the previous fflonk revision.

2024-12-04 08:51 davxy: should be fixed. New reference ark-ec-vrfs rev: d90e180

2024-12-04 13:26 davxy: A smaller value for tiny makes sense BTW. I think we can change it

2024-12-04 15:53 davxy: That will eventually be included in the vectors used to verify the block header. Safrole vectors are responsible for validating the Safrole STF.

2024-12-05 11:21 luke_fishman: Thank you, and apologize if I'm being pushy, but do you have a rough ETA for that? we in Jamxiir would really like to get some validations that we are signing/verifying correctly.

2024-12-05 14:56 danicuki: > <@danicuki:matrix.org> for no_assurances_with_stale_report-1 vector, how can the cores[0] report be nullified on the next state if there are no assurances in the block (thus no new report available)? I am referring to Formulas 11.15 and 11.16 - am I missing something? davxy afk: can you help with this issue?

2024-12-05 15:09 danicuki: Also, from what I noticed, vectors are considering a report staled when Ht = ρ†[c]t + U, but the GP formula says: af [c] ⇒ ρ†[c] ≠ ∅∧Ht ≤ ρ†[c]t + U (or vectors are using a value of U different then 5)

2024-12-05 15:29 danicuki: https://www.youtube.com/watch?v=9EcjWd-O4jI - Blockcowboy nostalgic reference

2024-12-05 15:46 davxy: > <@danicuki:matrix.org> Also, from what I noticed, vectors are considering a report staled when Ht = ρ†[c]t + U, but the GP formula says: > > af [c] ⇒ ρ†[c] ≠ ∅∧Ht ≤ ρ†[c]t + U > > (or vectors are using a value of U different then 5) Yeah looks like GP prescribes lazy deletion of stale reports. I asked @gav:polkadot.io for confirmation. Also I was considering a report stale if H_t >= ... . Looks like should be H_t > ... instead. Probably I need to fix these assurances vectors.

2024-12-05 22:08 danicuki: davxy afk: I just saw the new statistics vectors. Why do you use kappa_prime in the pre_state? shouldn't it simply be { pre_state: { kappa: {}} post_state: { kappa: {}} } kappa_prime in the pre_state seems very odd, and kappa_prime in the post_state seems redundant, as everything in post_state is prime :-)

2024-12-06 19:24 rustybot: I think we already discussed about this. Not sure if with you or someone else. - "input" is stuff found in the block: extrinsic and header - "state" are the relevant bits found in the global state relevant to push forward the STF. Said that, - some of these state bits may be "prime", "dagger" or double dagger (see asn.1 for semantics) - some of these bits are changed by the STF, while some others not. This may result in data redundancy. But otherwise I had to introduce a different struct for posterior state. I found this easier. (edited)

2024-12-07 11:18 olanod: Hello, don't know if it's of interest or too soon ... while y'all work hard on JAM implementations I'm getting interested in looking a bit higher up the stack(services) also following my interest of lower level embedded Rust(specially riscv). I've been toying around with a simple embedded/virtual OS that allows writing simple ink-like apps that can run on a riscv microcontroller(also linux, web and of course I also checked building against the pvm target ;)). I think I get now how to port this to run as a JAM service but the random idea I came to ask is to see if there would be interest in a **"pocket JAM"**, I'll buy this dev kit to experiment porting the os, it's interesting that has 3 different risc-v core including one rv64gcv and a rv32emc, I feel it would have enough juice to run a light client.

2024-12-07 21:46 gav: > <@davxy:matrix.org> Yeah looks like GP prescribes lazy deletion of stale reports. I asked @gav:polkadot.io for confirmation. > Also I was considering a report stale if H_t >= ... . Looks like should be H_t > ... instead. Probably I need to fix these assurances vectors. Reports are now actively removed on expiration: https://github.com/gavofyork/graypaper/pull/160/commits/3e99734afc3770b145539f20b1cfa2939bbecd80

2024-12-08 11:09 sourabhniyogi: Looking closely at the shape of CE134: Work-package sharing and CE135: Work-report distribution it appears that there continues to be an expectation that a guarantor has at most ~2 seconds to do its work eg "The third guarantor should be given a reasonable amount of time (e.g. two seconds) to produce an additional signature before the guaranteed work-report is distrubuted." Is the max gas allotted to refine going to be matched to this ~2s expectation? How does JAM's refine gas limit compare to the gas/weight available in a single block of Polkadot (Asset) Hub with revive-pallet? (edited)

2024-12-08 11:25 gav: There's no need to wait until the WR is computed before distributing the WP.

2024-12-08 11:26 gav: You just need to check the authorisation is reasonable.

2024-12-08 11:26 gav: And there's no need to synchronise with the block production.

2024-12-08 11:26 gav: The anchor block can be (IIRC) up to 8 blocks from the head of the chain at time of reporting. (edited)

2024-12-08 11:27 gav: JAM's gas limit is designed such that the WP should be able to be fetched and execute easily within the 6 seconds, the provisional gas limit is tuned to 5 seconds. (edited)

2024-12-08 11:41 sourabhniyogi: For both polkatool (usable to build contracts with refine-pallet, now on Westend Hub) and the JAM Rust SDK, there is this idea that arbitrary Rust code can be put into contracts and JAM Services. But in practice, in both cases, it appears there is a pretty serious "no\_std" requirement ... that makes it hard to put Rust packages using std into PolkaVM Contracts or JAM Services. They quickly hit this "error\[E0463\]: can't find crate for std"! This leaves us reaching out for a host function and a 'ecalli ___` as a "cheat code" to avoid rewriting the underlying dependencies on std packages, and thus get a revive-pallet based contract and/or JAM Service. Is there some trick that can be employed to overcome the no\_std limitation? (edited)

2024-12-08 11:44 gav: No. It’s there by design. This is consensus code.

2024-12-08 11:44 gav: You get trustless bare metal. Not Windows. (edited)

2024-12-08 12:14 sourabhniyogi: Its amazing how quickly the "you can run Doom with PolkaVM!" for the cameramen devolves into "how does info! work" 😅 How should non-Rust JAM implementations interface with Rust JAM SDK info! calls like these -- not talking about Windows here, just like printing stuff with caveman debugging -- is there a way? (edited)

2024-12-08 13:41 gav: Yeah - there’s an additional (unofficial) host call provided by polkajam which the logging system uses if enabled. (edited)

2024-12-08 13:42 gav: You can derive the details from the code in jam-pvm-common crate quite easily. Here's the relevant snippet:

2024-12-08 13:42 gav:

	#[polkavm_import(index = 100)]
	pub fn log(
		level: u64,
		target_ptr: *const u8,
		target_len: u64,
		text_ptr: *const u8,
		text_len: u64,
	);

2024-12-08 13:43 gav: So that takes 5 registers

2024-12-08 13:44 gav: The first is the log level, 0...5 which gives the severity of the item (edited)

2024-12-08 13:45 gav: then two UTF-8 strings passed as the usual offset/length combinations, the first for the subsystem name and the second for the message.

2024-12-08 13:45 gav: The subsystem name can by the null ptr, in which case it is in a "global" context without a specific subsystem.

2024-12-08 13:46 gav: I can copy/paste this into a JAM-community document.

2024-12-08 13:47 gav: Happy to consider other such unofficial host-calls if they make sense, though beware they'd only be for testing/debugging unless detailed in the GP.

2024-12-08 13:49 gav: There should be nothing which introduces non-determinism into the PVM, which basically means they have to be pure functions that could possibly have some useful side-effects (as long as they're transparent from the perspective of the GP).

ecdsa_recover=101
ecdsa_to_eth_address=102
hash_blake2_128=103
hash_blake2_256=104
hash_keccak_256=105
hash_sha2_256=106
sr25519_verify=107

2024-12-08 18:08 gav: > <@gav:polkadot.io> I can copy/paste this into a JAM-community document. https://hackmd.io/@polkadot/jip1

2024-12-09 06:34 weigen: Hello, I have a question about work package, is a work package limited to a maximum of four segment roots?

2024-12-09 11:28 gav: > <@weigen:matrix.org> Hello, I have a question about work package, is a work package limited to a maximum of four segment roots? No

2024-12-09 11:56 oliver.tale-yazdi: doc is private

2024-12-09 12:08 amritj: In CE 144 Audit announcement of JAMNP, how can we verify the announcement of no show auditors without the full list of reports they announced to audit? https://github.com/zdave-parity/jam-np/blob/main/simple.md#ce-144-audit-announcement (edited)

2024-12-09 18:55 gav: (a) could be done trivially without the trouble of defining a multi client standard host function to implement. (b) I see little point of this other than for code deduplication. In that case if it’s a serious issue we should allow for code-importing in a more sensible way.

2024-12-10 00:40 celadari: Hi, I would like to ask some help debugging signatures in work report tests. In tiny/multiple_reports-1.json (https://github.com/davxy/jam-test-vectors/blob/polkajam-vectors/reports/tiny/multiple_reports-1.json), our signatures are failing. **Could someone with a passing test vector share the following in HexString please ?** - The encoding of the first workReport - The hash of that encoding I post the full workReport definition in the thread to keep this message brief in the main chat. Thanks!

{
                "report": {
                    "package_spec": {
                        "hash": "0x63c03371b9dad9f1c60473ec0326c970984e9c90c0b5ed90eba6ada471ba4d86",
                        "length": 12345,
                        "erasure_root": "0x9fc7e637969aef1a95dfb560a914cf5161a76498db3aa19df131cea199ed6e44",
                        "exports_root": "0x5c9dca800c145685f052ce1ba343e2b66b4c333ee4c4ca3b29cb347b1009cb83",
                        "exports_count": 3
                    },
                    "context": {
                        "anchor": "0x39cb518983b02695034b3b92cb31a7334e1a2ec3ef7dbfa32c68e4e8444363f1",
                        "state_root": "0xd8c577816b629241676502d0461e4eae42a375461314d64484f35f4228da23d6",
                        "beefy_root": "0x71c150347dc035847d5944c864a2d3ed0fa410341b2ffbfdeeaa1c9d6aa9cacd",
                        "lookup_anchor": "0x16bda47e5a68daf53c39ddee8af4ecaced7e87f3f0ac9da5a6f4f9e41350d319",
                        "lookup_anchor_slot": 6,
                        "prerequisites": []
                    },
                    "core_index": 0,
                    "authorizer_hash": "0x9a3a97d1950356ef6d3c20acb5ab6699be454b1498ecd513bdc6d849497e42eb",
                    "auth_output": "0x",
                    "segment_root_lookup": [],
                    "results": [
                        {
                            "service_id": 42,
                            "code_hash": "0x6470fd21983eae8d706f1edd5e2dc5afe095980f8fb7bd4ebfd33550d8730246",
                            "payload_hash": "0x3cf2d09da453bbdd6b68d8a1acd5f15ba23ba46d4ff087592b3d319432500a50",
                            "accumulate_gas": 10000,
                            "result": {
                                "ok": "0x64756d6d792d726573756c74"
                            }
                        }
                    ]
                },
                "slot": 14,
                "signatures": [
                    {
                        "validator_index": 1,
                        "signature": "0xeef064e3973b71fa4b0ce0c69c6799cc9b0e1887a894a93559a4c3a3848f039df82de95c70218588134ac51c2a329936ec54bc320b4c97c5f29f46acd1f4860e"
                    },
                    {
                        "validator_index": 4,
                        "signature": "0x68e84d7253380e8beb1fb71a54a8a0a805c06ea756da74d79bcbf5b811306cea560f27abaca175f6908ab54e4acc7bb919ab87ac616e7df106d3b043b71bac04"
                    },
                    {
                        "validator_index": 5,
                        "signature": "0x2c369533280dcd2fbdc2efeaf9dd0e042363ad14a4dee64a0f539def799f488912c85743132c5966bf5f9e65ec4d13cda5281fbbbe0c705a22e00424c858bf0b"
                    }
                ]
            },

2024-12-10 11:02 gav: IMG_3845.jpeg

2024-12-10 11:03 gav: Here’s a simple diagram of it.

2024-12-10 11:04 gav: So the first proof page would be formed as p0 with h1 h2,3 and h4,7

2024-12-10 11:04 gav: The second would be p1 h0 h2,3 h4,7

2024-12-10 11:05 gav: The fifth would be p4 h0,3 h5 h6,7

2024-12-10 11:05 gav: And so on.

2024-12-10 11:06 gav: Each page is made up of 64 segments, each hashed with the proper prefix. This is then appended with the subtree path proof (ie the complementary nodes) from the segment root’s first child down to the root of the subtree holding the 64 segments of the page. (edited)

2024-12-10 11:09 gav: This is enough information to form a proof of inclusion for any segment in the page given the segment root commitment.

2024-12-10 13:26 weigen: Thanks for the clear explanation!

2024-12-10 18:12 danicuki: davxy afk: What value are you using for G_A on the high_work_report_gas-1 test? This tests seems to be over the total gas, but no error expected in the vector. The sum in work results is 10_000_000, much above the 100_000 limit (or you are using another value for G_A).

2024-12-10 18:37 jaymansfield: > <@danicuki:matrix.org> davxy afk: What value are you using for G_A on the high_work_report_gas-1 test? This tests seems to be over the total gas, but no error expected in the vector. The sum in work results is 10_000_000, much above the 100_000 limit (or you are using another value for G_A). This might help: https://github.com/w3f/jamtestvectors/pull/20#issuecomment-2526203035

2024-12-10 19:57 celadari: Regarding work report guarantees test vectors (https://github.com/davxy/jam-test-vectors/blob/polkajam-vectors/codec/data/guarantees\_extrinsic.json) => is it normal that alpha gets updated ? (field "auth_pools" is not the same in pre_state and post_state) I ask this because I know alpha should be updated using phi' and E_G but it should be done after accumulation, more precisely after computing phi' ? (edited)

2024-12-11 00:33 celadari: I'm gonna put here a few questions I have regarding work report guarantees test vectors (https://github.com/davxy/jam-test-vectors/blob/polkajam-vectors/codec/data/guarantees_extrinsic.json)

2024-12-11 00:35 celadari: - I think there might be an issue with file full/consume_authorization_once-1.json because test vector passes if I don't sort the hashes when computing super peak but fails if I do. However, if I refer to the GP 0.5.2 - equation (E.10) => hashes should be sorted to compute super peak

2024-12-11 00:38 celadari: I'm actually gonna post them on the github PR, might be the right place for it (edited)

2024-12-11 08:48 danicuki: I don't see anywhere in the formula that the elements should be "sorted". They need to be "consumed" in the order they are served , but not sorted (edited)

2024-12-11 08:50 dakkk: I answered 4 of 5 on gh

2024-12-11 11:10 clearloop: may I ask about the progress of erasure\_coding, saw there is still a lot of TODOs in the implementation that generates the testvectors, shall we push the testvectors together or it's enough for our current stage? (edited)

2024-12-12 08:24 danicuki: About the test vectors, I would suggest to include in the post_state only the fields that are supposed to be tested. Including fields with incorrect post transition value can create confusion for us. Or explicit post somewhere else that these values should not be checked. (edited)

2024-12-12 12:50 davxy: > <@danicuki:matrix.org> About the test vectors, I would suggest to include in the post_state only the fields that are supposed to be tested. Including fields with incorrect post transition value can create confusion for us. Or explicit post somewhere else that these values should not be checked. What do you mean with: > include in the post_state only the fields that are supposed to be tested. And > Including fields with incorrect post transition value can create confusion for us. ?

2024-12-12 12:53 davxy: For the first statement I imagine you meant having a different struct for the posterior state?

2024-12-12 12:54 danicuki: > <@davxy:matrix.org> For the first statement I imagine you meant having a different struct for the posterior state? No. Same structure

2024-12-12 12:55 davxy: So now it is even less clear. Can you elaborate?

2024-12-12 15:00 tvvkk7: Hi, I would like to confirm the result of work report/work returned from PVM invocation. The result of the work report will be "ok" , "panic" or "oog" as in jamtestvectors ? The result codes of innver PVM invocation such as HALT=0 is for PVM itself ? (edited)

2024-12-12 16:04 ycc3741:

2024-12-12 16:08 ycc3741: davxy afk: When I used the validator from your "jam-test-vectors" library , the following error occurred. I suspect that something might be wrong: (edited)

2024-12-12 16:08 ycc3741: 螢幕擷取畫面 2024-12-13 000742.png

2024-12-12 17:34 davxy: > <@ycc3741:matrix.org> davxy afk: When I used the validator from your "jam-test-vectors" library , the following error occurred. I suspect that something might be wrong: Are you using this fork? https://github.com/davxy/asn1tools/

2024-12-12 17:54 celadari: We believe it would be helpful to have a miscellaneous field that is not used for comparing expected_post_state and pre_state, but instead provides intermediary state values. This would make it easier to identify which parts failed when test vectors fail. For example, in the case of work reports, the miscellaneous field could include guarantor assignments and previous guarantor assignments. This way, if someone encounters a failing test vector, they could check whether the issue is related to these assignments or something else. **What is everyone opinion on that ?** (edited)

2024-12-12 18:09 sourabhniyogi: To make our own debugging life easier and to help teams compare state with others we are extending our representation of serialized (k,v) state from just

            [
                "0x00fe00ff00ff00ff4105beb468bf2d38812fae29cac024f6bf1fd208b5adee5b",
                "0x00000000000000000020009b0000000000808628110000002810000000281c000000287333073200140a0400fffe00000000330b24330732008b11f8711014070000fffe0000000076782033090020330a0010330b0030330c00400a09140b00e0fdfe00000000480b70b704140904e0fdfe00000000330804330a04140700e0fdfe000000000a038d77208e772078108b110832003307320021842a80540a2011110550024802282915",
            ],

            [
                "0xff00000000000000000000000000000000000000000000000000000000000000",
                "0xee4105beb468bf2d38812fae29cac024f6bf1fd208b5adee5b4bb7c7f85f60ba102700000000000064000000000000006400000000000000fb0000000000000002000000",
            ]

            [
                "0x7fc1b2006200c10015b4b5bb2060104268e1155884d8b5ab2aad3c12475c7e95",
                "0x01cfaf4b00",
            ]

to (k, v, type, k\_and\_v\_metadata)

            [
                "0x00fe00ff00ff00ff4105beb468bf2d38812fae29cac024f6bf1fd208b5adee5b",
                "0x00000000000000000020009b0000000000808628110000002810000000281c000000287333073200140a0400fffe00000000330b24330732008b11f8711014070000fffe0000000076782033090020330a0010330b0030330c00400a09140b00e0fdfe00000000480b70b704140904e0fdfe00000000330804330a04140700e0fdfe000000000a038d77208e772078108b110832003307320021842a80540a2011110550024802282915",
                "account_preimage",
                "s=0|h=0xee4105beb468bf2d38812fae29cac024f6bf1fd208b5adee5b4bb7c7f85f60ba|plen=170"
            ],

            [
                "0xff00000000000000000000000000000000000000000000000000000000000000",
                "0xee4105beb468bf2d38812fae29cac024f6bf1fd208b5adee5b4bb7c7f85f60ba102700000000000064000000000000006400000000000000fb0000000000000002000000",
                "service_account",
                "s=0|b=10000 g=100 m=100 l=251 i=2|clen=32"
            ]

            [
                "0x7fc1b2006200c10015b4b5bb2060104268e1155884d8b5ab2aad3c12475c7e95",
                "0x01cfaf4b00",
                "account_lookup",
                "s=3244470911|h=0xc4254ce54e9166133e4b84446c88bbce6d31324bbac003e7c72d39b4918a8d02 l=193 t=[4960207]|tlen=1"
            ]

This is valuable not merely because {s, h, ... } is "lossy" going into k, but because we can quickly reason about (really, grep for) keys, values over many states quickly. This makes it easy for us to settle debates within our team quickly and others if we all post something similar. Having 2 extra values in the k,v serialization is simpler than putting it in more verbose pre-serialized state representation. We're not super wedded to any specific metadata format for the 4 service k,v - but prefer non-JSON to have simple "cut" work rather than complex json attribute extraction (so, as you can see above, |-separated, space separated, then =-separated). What do you think about this? (edited)

2024-12-12 18:49 ycc3741: I am using this: https://github.com/w3f/jamtestvectors (edited)

2024-12-12 19:52 davxy: > <@ycc3741:matrix.org> I am using this: https://github.com/w3f/jamtestvectors No, what I mean is that the validate.py script relies on the asn1tools Python package. You need to use the version I shared with you, as it includes some modifications compared to the upstream version.

2024-12-12 20:11 davxy: My 2 cents regarding the test vectors format discussion. As the conversation seems to be getting somewhat scattered, I suggest moving it to a dedicated GitHub issue. Personally, (of course) I believe that the current one is the best format for the moment, but that's just my opinion. Naturally, **everyone has an opinion** and this can get fuzzy fast, especially in a chat. However, I'm not sure how effective it is in this context. I plan to finalize the collection of vectors for the different STFs using current format, as we are quite close to covering the full on-chain requirements. If we find a better format later, a switch can certainly be considered. Of course, anyone is free to propose independent vectors in other formats, as Jam Duna is already doing (although would be more beneficial to remain aligned. And that is the main purpose of the GH issue) (edited)

2024-12-12 20:22 sourabhniyogi: > <@davxy:matrix.org> My 2 cents regarding the test vectors format discussion. As the conversation seems to be getting somewhat scattered, I suggest moving it to a dedicated GitHub issue. Personally, (of course) I believe that the current one is the best format for the moment, but that's just my opinion. Naturally, **everyone has an opinion** and this can get fuzzy fast, especially in a chat. However, I'm not sure how effective it is in this context. > > I plan to finalize the collection of vectors for the different STFs using current format, as we are quite close to covering the full on-chain requirements. If we find a better format later, a switch can certainly be considered. Of course, anyone is free to propose independent vectors in other formats, as Jam Duna is already doing (although would be more beneficial to remain aligned. And that is the main purpose of the GH issue) Awesome - which issue? https://github.com/w3f/jamtestvectors/issues

2024-12-13 09:03 dakkk: a new one

2024-12-13 10:55 danicuki: Good morning all! I have doubt about a safrole vector "enact-epoch-change-with-padding-1" My gamma\_z (epoch root) is not matching the vector's - may be something related with the zeroed offender key. From what I understood, we should zero the offender key before calculating the ring commitment root. This zero is a 32 zero bytes. Did anyone could make this vector pass? gamma\_k (b):

["aa2b95f7572875b0d0f186552ae745ba8222fc0b5bd456554bfe51c68938f8bc",
"0000000000000000000000000000000000000000000000000000000000000000",
"5e465beb01dbafe160ce8216047f2155dd0569f058afd52dcea601025a8d161d",
"1ecc3686b60ee3b84b6c7d321d70d5c06e9dac63a4d0a79d731b17c0d04d030d",
"3d5e5a51aab2b048f8686ecd79712a80e3265a114cc73f14bdb2a59233fb66d0",
"7f6190116d118d643a98878e294ccf62b509e214299931aad8ff9764181a4e33"]

my gamma_z (doesn't match the vector):

81eaf0953ed7474a81fa5445616a141c4c1f54ebd3cd8bda932bf612fc7fa41a90ce8db077f4bc5a3f71336e98a57ea8a6385289d15586c18ff7f8f5661a00bb783afe07938431c77d8cccc3585612cf3461f987e01dadfec7aefbc2aa3bb31a92e630ae2b14e758ab0960e372172203f4c9a41777dadd529971d7ab9d23ab29fe0e9c85ec450505dde7f5ac038274cf

(edited)

2024-12-13 10:57 dakkk: you should use the bandersnatch padding point

2024-12-13 10:58 danicuki: > <@dakkk:matrix.org> you should use the bandersnatch padding point What is padding point? I am not aware of this? 🤔

[Ring Padding]
X: 23942223917106120326220291257397678561637131227432899006603244452561725937075
Y: 1605027200774560580022502723165578671697794116420567297367317898913080293877
Compressed: 0xf5399e03f2121ff4c5d33386cdc66d56a6c5132b739f753442f7bda6c7698c03

                "bandersnatch": "0x1ecc3686b60ee3b84b6c7d321d70d5c06e9dac63a4d0a79d731b17c0d04d030d",

2024-12-13 10:59 dakkk: checkhe note at the end of apx G

2024-12-13 12:30 ycc3741: Thanks a lot

2024-12-14 15:33 ycc3741: davxy afk: Hi I wonder whether we should use this one or this one for singature and verification if using the vector you provide to test. (edited)

2024-12-15 01:29 jam_man: Has phase one already been submitted by teams?

2024-12-15 06:19 davxy: https://github.com/davxy/bandersnatch-vrfs-spec (edited)

2024-12-15 07:17 clearloop: hi there! anybody feel weird about the tests cases (tiny) related to formula 11.23 (11. reporting)?

// formula 11.23
let (e, k) = (tau' - R) / E == (tau' / E) ? (eta'2, kappa'): (eta'3, lambda')

most of the test cases (tiny) are using tau=14 in both guarantee & reports, however, tau=14 meets the case using (eta'3, lambda') instead of the eta'2 pair while the current tiny test cases are expecting the eta'2 pair to be used (edited)

2024-12-15 10:50 prematurata: AFK now but check README. R is different for tiny

2024-12-15 10:57 clearloop: yes but still...

let (e, k) = if (14 - 4) / 12 == 14 / 12 ? (eta_2, kappa): (eta_3, lambda)
               = if 0 == 1  ? (eta_2, kappa): (eta_3, lambda)
               = (eta_3, lambda)

I can pass all test vectors via ignoring /E but I know it is not correct ))) and if I stick to this formula over 10 test cases will get bad signature xd

2024-12-15 10:58 prematurata: will get back in afew and can check if you want.... can you name one specific test?

2024-12-15 11:00 clearloop: yes! for example https://github.com/davxy/jam-test-vectors/blob/polkajam-vectors/reports/tiny/many_dependencies-1.json

2024-12-15 14:27 prematurata: 'm using p\_eta3 as well and test is passing (edited)

2024-12-16 11:14 prematurata: davxy afk: can you tell me which lib you using to sign/verify bandersnatch?

2024-12-16 19:00 ycc3741: davxy afk: I want to use the w3f Rust library, but we’re implementing our project in Go. We’re wondering if it’s acceptable to write a Rust file to encapsulate your Rust functionality, then compile it for FFI so that we can use it in Go. What I mean is: would doing this be acceptable according to the rules, or would it violate any regulations about which implementation languages are allowed?

2024-12-16 19:25 davxy: acceptable (see rule 1 https://jam.web3.foundation/rules ) (edited)

2024-12-16 19:29 ycc3741: Thanks a lot (edited)

2024-12-17 18:37 davxy: I've opened an issue to gather opinions regarding the detection of duplicate packages in the "reports" STF vectors. https://github.com/davxy/jam-test-vectors/issues/7

2024-12-17 18:38 davxy: Mostly to properly address https://github.com/w3f/jamtestvectors/pull/28#issuecomment-2535457582

2024-12-17 18:39 davxy: Please leave your feedback directly in the GH issue. Thank you.

2024-12-17 20:03 prematurata: heads up for anyone who relied on the PR 14 to implement trie https://github.com/w3f/jamtestvectors/pull/14#issuecomment-2549423040. Although as stated in the comment the implementation makes more sense than what has been written in the Graypaper, I believe that PR is out of spec. Also considering several implementors said they aligned with the PR provided vectors (which again is not in spec of gp) made me appreciate even more the clean room implementation rule. In this specific case the implementation was provided along with the vectors which I'd say should be discouraged.

2024-12-18 15:15 fbranciard: Hi, I have 3 questions about Core Play vision and services compositions: 1- I wonder if, in a sense, it will look like CRDT (Conflict-Free Replicated Data Type) world state. Or it will not be at this level but rather at a low level like several VMs: (start, stop, resume, VM state) with ability to communicate together through exposed APIs. 2- Another question: is Solid (from Tim Berners-Lee), separation of concerns - data, app, identity - will be a guideline in the design of Core Play? 3- Last question: I envision here https://forum.nextgraph.org/#narrow/stream/3-devs/topic/ISCC.20metadata.20on.20Document/near/776 that Jam could be a notary anchor for a Document-based system, is that correct?

2024-12-18 21:19 gav: CorePlay idea is based around actors whose memory exist within DA, but are committed to on-chain. Async and sync interaction patterns are possible via message queuing and typical async programming techniques, as well as co-scheduling. To the outside world (like developers and blockchain inspectors) it’ll look a lot like a regular smart contract environment except that the contracts are actually actors and execute perpetually. I’m not knowledgable about these of TBL’s musings but given his other work cannot imagine they’re especially relevant. (edited)

2024-12-18 21:22 gav: The forum post you link isn’t especially helpful but from what I can tell it is a DAG project without any definitive global state. JAM has global state albeit with a slightly weakened coherence compared to fully synch systems so I couldn’t imagine the architecture is comparable.

2024-12-18 21:23 gav: CRDT has no relevance to CorePlay.

2024-12-18 21:54 fbranciard: Thanks for the explanation, looking forward to see jam services.

2024-12-19 11:13 subotic: A heads up for those that are waiting for the 64-bit PVM test vectors. Here is my PR: https://github.com/paritytech/polkavm/pull/238 adding 64-bit test cases. Jan still needs to do the review, so no guarantees that everything is correct. Since Jan is now on his well deserved holiday break, it will be some time until this gets merge. If you have any questions or problems, don't hesitate to ask.

LOAD_IMM_64 = 20 // 0x14
LOAD_I32 = 57    // 0x39
LOAD_U64 = 58    // 0x3a
STORE_IMM_IND_U64 = 73 // 0x49
STORE_IND_U64 = 113 // 0x71
LOAD_IND_I32 = 119  // 0x77
LOAD_IND_U64 = 120  // 0x78
CMOV_NZ_IMM = 138   // 0x8a
ADD_IMM_64 = 139 // 0x8b
SHLO_L_IMM_64 = 141 // 0x8d
SHLO_R_IMM_64 = 142 // 0x8e
DIV_U_64 = 183	// 0xb7
DIV_S_64 = 184	// 0xb8
REM_U_64 = 185	// 0xb9
REM_S_64 = 186	// 0xba
SHLO_L_64 = 187	// 0xbb
SHLO_R_64 = 188	// 0xbc
SHAR_R_64 = 189	// 0xbd
MUL_UPPER_S_S = 193 // 0xc1
MUL_UPPER_U_U = 194 // 0xc2
MUL_UPPER_S_U = 195 // 0xc3
CMOV_NZ = 199   // 0xc7

2024-12-19 19:01 sourabhniyogi: It appears that this MMB ref was NAYed to oblivion -- https://polkadot.polkassembly.io/referenda/1317 What does this mean for our MMB vs MMR implementation / Beefy life?

2024-12-19 19:17 gav: > <@sourabhniyogi:matrix.org> It appears that this MMB ref was NAYed to oblivion -- https://polkadot.polkassembly.io/referenda/1317 > What does this mean for our MMB vs MMR implementation / Beefy life? Nothing.

2024-12-19 19:27 gav: “At least for now” everything is “effectively done” except what is on milestone 5. (edited)

2024-12-20 05:17 sourabhniyogi: What is the expected plan for Grandpa Messages https://spec.polkadot.network/chap-networking#sect-msg-grandpa ?

2024-12-20 05:20 clearloop: May I ask if there is test data like a sequence of blocks ( from genesis ) for testing block importing? (edited)

2024-12-20 22:24 rgekfd3e: What are the improvements in dApp UX design when using JAM instead of an EVM dApp?

2024-12-21 13:59 gav: > <@rgekfd3e:decred.org> What are the improvements in dApp UX design when using JAM instead of an EVM dApp? JAM is far more general a platform than EVM and doesn’t put any limitations on user-interaction directly. All of the stuff that Ethereum is trying to work around due to design limitations in its transaction model don’t apply to JAM (or, indeed, to Polkadot).

2024-12-21 16:25 wind.sixwings: signing_context_example.png

2024-12-21 16:25 wind.sixwings: When reading the GP ch6, I saw that (6.18) ~ (6.20) defined 3 Signing Contexts. I would like to ask whether these 3 Signing Contexts need to be defined by the implementation team themselves or are there other official sources of definition? Another question raised by team members during the discussion is whether Signing Contexts are constant in most cases? Or will it be modified under certain circumstances?

# the singing context is "jam_entropy"
X_E = $jam_entropy 

2024-12-21 20:25 gav: Define everyday users. Define interact.

2024-12-21 20:28 gav: Coreplay might reasonably be an upgrade path from pallet revive. The rest of Polkadot could interact with it synchronously via co-scheduling and co-integration, or asynchronously in the usual way via XCM. Non Polkadot systems could interact using a bridge. Coreplay is an idea at present; there is no writeup or lecture.

2024-12-21 23:10 gav: It’s not exactly an upgrade for pallet-revive but it might provide an upgrade path for whatever systems are using it. We’ll have to see.

2024-12-21 23:11 gav: Taken at your definitions then yes it would be interacted with by everyday users. (edited)

2024-12-21 23:11 gav: However I think those definitions are not especially helpful as they are so broad.

2024-12-21 23:13 gav: That transactions get signed in a dapp is about as relevant as the fact that a TCP packet gets authored when fetching a webpage. We don’t say that people “send TCP packets” though. (edited)

2024-12-22 20:33 gav: 2. As I already answered in the fellowship call a few days ago, no.

2024-12-22 20:34 gav: 3. Not much. There are already fellows who are experts in one area and not others.

2024-12-22 20:35 gav: 4. I don’t see all that much need for JAM development from the “general public”. It’s a complex protocol not Roblox.

2024-12-22 20:36 gav: 1. I’ve no idea beyond the obvious “build it and they will come” mantra. Support of Polkadot is no different to support of JAM, since JAM will become Polkadot. Thus you may as well ask that question in the Polkadot forum but with “JAM” exchanged for “Polkadot”. No. (edited)

2024-12-22 20:41 gav: Finally @sourabhniyogi:matrix.org, please do not treat this room as a perpetual Gavin AMA. If you have something to contribute to the conversation, contribute it. If you wish to start a new topic do so. But quizzing me on non-technical, financial strategy is totally unacceptable and goes against the CoC. I’ve already warned you of this. (edited)

2024-12-22 20:42 gav: I thought I had already made this clear to you. Apparently not. Consider this a final warning. (edited)

2024-12-23 13:36 clearloop: as we can see in the jam-types spec, https://github.com/w3f/jamtestvectors/blob/master/jam-types-asn/jam-types.asn#L32, the bls public key has 144 bytes, if I'm not mistaken, bls\_12\_381 has 96 bytes or 48 bytes (compressed) in the public key, that this 144 bytes could be the combined version of the compressed and the uncompressed versions? if so, shall we place the compressed version ahead of the uncompressed version since it could be more frequently used? (edited)

2024-12-23 15:57 davxy: copy/paste of an old reply given in the GP chan some time ago: The public key for the scheme consists of two public elements, each corresponding to one of the two groups used by BLS for pairing: G1 and G2. Specifically, it is represented as (g1 sk, g2 sk), where g1 is the generator of G1 and g2 is the generator of G2. Each point in G1 is compressed into 48 bytes, while each point in G2 is compressed into 96 bytes. This scheme allows both fast verification of aggregate signatures and (pairing less) single signature verification. The full scheme is detailed in this paper: https://eprint.iacr.org/2022/1611 and is implemented by this library: https://github.com/w3f/bls. (edited)

2024-12-25 15:37 jaymansfield: Merry Christmas everyone! As we refine and accumulate memories this season, let’s celebrate connection, innovation, and a unified vision for the future!

2024-12-25 16:29 terryhung: Hi davxy afk , Merry Christmas! I want to test the data in bandersnatch_ed_sha512_ell2_ring.json via ring_vrf_verify flow. I’d like to understand how to construct the ring from this data and how the Verifier should output ring_pks_com. Here is the related issue I opened: https://github.com/davxy/ark-ec-vrfs/issues/39 thanks!

2024-12-26 17:17 tvvkk7: 螢幕擷取畫面 2024-12-27 011630.png

2024-12-26 17:17 tvvkk7: Hello, I have a problem with deserialization while implementing Eq. 6.26. I used the deserialization of Equation C.5 with l=4 to compute the result, but got different result with the testvectors. Am I using the wrong deserialization and which term of deserialization should I use?

2024-12-27 01:55 luke_jamixir: > <@tvvkk7:matrix.org> Hello, > > I have a problem with deserialization while implementing Eq. 6.26. I used the deserialization of Equation C.5 with l=4 to compute the result, but got different result with the testvectors. Am I using the wrong deserialization and which term of deserialization should I use? We also used to have problems there and it turned out to be a problem with our H4(blake hash 4 bytes) We also deserialize using l=4 so that's likely not your issue

2024-12-27 04:00 tvvkk7: Got it! Thanks

2025-01-01 12:03 0xjunha: Hey guys we're in JAM COMMON ERA!

2025-01-01 20:29 tvvkk7: Hello, I have a problem on slot key sequence(Eq. 6.24) and the fallback mode happens. In the condition, gamma\_s' will be updated by the F function. I got the same result using testvectors of W3F, but got different result using test data of jam-duna. I use eta\_2 in epoch 425523\_011 (prior slot key sequence). I use gamma\_s' in 425523\_000 (posterior key slot sequence). According to Eq. 6.24, in the first slot, gamma\_s' will be updated using eta\_1. The other slots wii be gamma\_s' = gamma\_s if e' = e. So, the result should be gamma\_s' is finally updated using eta\_1. Is my understanding incorrect ? (edited)

2025-01-02 02:23 shwchg: the files in state_snapshots is the post_state after applying the corresponding block. If you would like to use it as the pre_state for the next block, you should use eta_0

2025-01-02 10:11 tvvkk7: Thanks a lot

2025-01-03 09:33 vinsystems: Will JAM be able to support distributed validator technology?

2025-01-03 10:47 gav: Define "distributed validator technology"

2025-01-03 10:48 clearloop: hmm, interesting, in my implementation, the validator could be local and remote, however, I think this kind of stuff is not proper for the current stage since we need to make the network work first XD (edited)

2025-01-03 11:12 vinsystems: Basically I mean the same (or similar) approach used in Ethereum, which spreads out key management across multiple parties to reduce single points of failure. (edited)

2025-01-03 11:46 gav: JAM makes no great requirements on how validators manage/store their secret keys, only that they provide an IPv6 endpoint and set of relevant public keys. (edited)

2025-01-03 11:47 gav: If you want to give a better definition than "will it be like Ethereum?", I'm happy to offer a more detailed commentary.

2025-01-04 04:36 jakechen: Hello, we are encountering an issue with header serialization. When the test data we use does not include epoch_mark and tickets_mark, we can correctly serialize the header data and compute the parent header hash for the next time slot using BLAKE2b-256. However, when the test data includes either epoch_mark or tickets_mark, the hash we calculate does not match the test data. In addition to handling the discriminator encoding (Eq. C.9), does the serialization of epoch_mark and tickets_mark require any additional processing for the array data, such as epoch_mark.validators? We are troubleshooting using the test data files 425530_000.json and 425530_001.json.

2025-01-05 11:20 gav: > <@amritj:matrix.org> In graypaper here https://graypaper.fluffylabs.dev/#/911af30/19e30019e600 > > It is mentioned that import spec could contain both segment root and work package hash and to distinguish between them the work package hash value are tagged > > > But here according to jam official codec it is just a hash: > > https://github.com/w3f/jamtestvectors/blob/90fcf9020fa269a3aecc23feac0d09e7fee9123b/jam-types-asn/jam-types.asn#L121 > > So, how do we identify, if hash is tagged or not i.e. is it a segment root or work package hash? > > > There’s a proper encoding given in C.29

2025-01-05 14:58 tvvkk7: Hello, I wold like to make sure whether the condition will happen if the report is good, but the work report hash is not in the fault. As gray paper defines good reports should be in the fault If any good report is not in fault, do we need to take any measure like append the work report hash to fault ? (edited)

2025-01-05 14:59 tvvkk7:

2025-01-05 18:00 gilescope: C.6 Are there any test vectors for the length encoding? Is number 12 really encoded as 0x30 as some test vectors suggest? Sorry for asking but something's not quite right and I'm soaking up a lot of time on C.6.

2025-01-06 07:15 gav: > <@tvvkk7:matrix.org> Hello, I wold like to know whether the condition will happen if the report is good, but the work report hash is not in the fault.(Eq. 10.13) If this happens, do we need to take any measure like append the work report hash to fault ? Best to provide a link with the gray paper reader.

2025-01-06 07:16 gav: > <@gilescope:matrix.org> C.6 Are there any test vectors for the length encoding? Is number 12 really encoded as 0x30 as some test vectors suggest? Sorry for asking but something's not quite right and I'm soaking up a lot of time on C.6. Which test vectors imply that?

2025-01-06 10:10 gilescope: > <@gav:polkadot.io> Which test vectors imply that? https://github.com/w3f/jamtestvectors/blob/dc20cbce7d855974aa64301a4d952e22f277010f/safrole/tiny/skip-epoch-tail-1.json#L169 - the gamma_a is a variable length and is 12 long according to the json for the pre_state. But the byte is 0x30 when I look at the hex immediately before 11da6d1f761ddf9bdb4c9d6e5303ebd41f61858d0a5647a1a7bfe089bf921be9 in the corresponding scale file.

2025-01-06 10:26 gav: That's probably an old test vector: 0x30 is a SCALE Compact integer encoding for 12.

2025-01-06 10:28 gav: Best to use the vectors in https://github.com/w3f/jamtestvectors/pull/28

2025-01-06 10:30 xlchen: .scale is using the old/non-gp scale codec. you should reference .bin files

2025-01-06 11:29 piotrzwolinski:

2025-01-06 11:30 piotrzwolinski: Hello JAMmers! At Fluffy Labs, we build tools like the Gray Paper Reader and PVM Debugger to support JAM development, and we’d love your feedback! 🙌 👉 Anonymous feedback: https://docs.google.com/forms/d/e/1FAIpQLSducTy9dgPd0FG0QyWH-QKuolf1Sfp8I8kI2vdkKghuu82JoA/viewform 👉 Non-anonymous testimonial: https://docs.google.com/forms/d/e/1FAIpQLSfP2Yf0cfcb5vtaNzqJb40i-H5_gVxeTpWCLactlSmPMydxiQ/viewform Thank you for helping us improve and support the JAM community! 🚀

2025-01-06 11:33 tvvkk7: Thanks, got it!

2025-01-08 04:19 clw0908: Hello, I have a question regarding the **codec** part in **jamtestvector**: I’m wondering whether the **"extrinsic_hash"** in **block.json** is calculated using **GP-0.5.3 equations 5.4, 5.5, and 5.6**, or if it is still calculated using the older version of the function. Since the calculation method hasn't been modified for a long time, I’d like to confirm. The commit that added block.json was made on **September 18, 2024**. Since then, there have been no further modifications to **"extrinsic_hash"**.

2025-01-08 09:17 tomusdrw: > <@clw0908:matrix.org> Hello, I have a question regarding the **codec** part in **jamtestvector**: > > I’m wondering whether the **"extrinsic_hash"** in **block.json** is calculated using **GP-0.5.3 equations 5.4, 5.5, and 5.6**, or if it is still calculated using the older version of the function. Since the calculation method hasn't been modified for a long time, I’d like to confirm. > > The commit that added block.json was made on **September 18, 2024**. Since then, there have been no further modifications to **"extrinsic_hash"**. It doesn't really matter. The codec test vectors are not meant to be semantically correct. They are just testing if you can decode the data given some schema.

2025-01-08 16:08 gav: I think it'd be good to request treasury support for 2 or 3 deliberately-produced JAM (services and authorizer) SDKs, aside from the SDK produced by Parity. The best way I see this of happening is to ask teams who have passed M1 to submit proposals for an SDK framework, giving code examples for the sort of thing their SDK would allow to write. I would then get some initial treasury funding of ~$100k, possibly just from the Fellowship Treasury. From the proposals, we could fund up to 10 teams ~$10k each to make initial prototypes of their proposals and 2 or 3 of the most promising prototypes could be further developed into proper SDKs. (edited)

2025-01-08 16:09 gav: Is there any interest from teams close to M1 to put forward such a proposal and build a prototype?

2025-01-08 17:37 jaymansfield: ima_2c6a278.png

2025-01-08 17:38 jaymansfield: Happy to share a glimpse of JavaJAM’s first testnet utilizing QUIC messaging. Still a lot of tweaks to go but feels good to get to this stage.

2025-01-08 18:11 clearloop: Hi there, we just introduced a simple general testvectors runner, please feel free to try it out!

cargo install specjam
specjam dummy

https://github.com/spacejam-network/specjam (edited)

2025-01-08 21:09 gav: > <@jaymansfield:matrix.org> Happy to share a glimpse of JavaJAM’s first testnet utilizing QUIC messaging. Still a lot of tweaks to go but feels good to get to this stage. Could you repost with a link someone other than here? Media inclusion doesn’t seem to work for me at all:(

2025-01-08 21:46 jaymansfield: > <@gav:polkadot.io> Could you repost with a link someone other than here? Media inclusion doesn’t seem to work for me at all:( Realizing now I probably should have made it a video, but you can find the same screenshot here: https://x.com/javajamio/status/1877069057372881158?s=46&t=WteYIn3tg7gE1pCvJh1G2Q

2025-01-13 19:02 tvvkk7: Hello, I have a question about serialization. We have to check the data types of the serialization target to choose which kind of the serialization we should use, is that right ? All the int types should be serialized using equation C.5. And only the up-down arrow serialization uses equation C.6 ? (edited)

2025-01-13 19:42 gav: Yes that’s right. We might see C.6 used a bit more in other parts of the protocol as time goes on but for now your reading is correct. (edited)

2025-01-13 20:03 ycc3741: thanks a lot. We've been a bit confused for a while.

2025-01-14 17:41 yu2c: Hello, does anyone know how to join this call ? I couldn't find any public invite information

2025-01-14 17:44 dakkk: good question

2025-01-14 17:52 rustybot: > <@yu2c:matrix.org> Hello, does anyone know how to join this call ? > I couldn't find any public invite information How was this call organized? Is there another channel dedicated to JAM that I am not aware of? I completely missed it

2025-01-14 17:55 ycc3741: We have no idea, either.

2025-01-14 17:55 jaymansfield: Also was not aware of any meetup (edited)

2025-01-14 18:01 rustybot: It seems plenty of teams were in the loop about this meetup (https://x.com/danicuki/status/1879210677216379008?t=VCKJxGk4SEqM1uRGC0IFcA&s=19). So either I've been living under a rock, or there's a communication channel that some of us aren't privy to

2025-01-14 18:42 prematurata: I didn't know as well

2025-01-14 18:51 oliver.tale-yazdi: It was in the JAM0 channel since it is not really canonical "Jam", just among implementors: https://matrix.to/#/!KKOmuUpvYKPcniwOzw:matrix.org?via=matrix.org&via=parity.io (not sure if you need an invite)

2025-01-14 19:08 yuchun: Could you invite me? thanks (edited)

2025-01-14 19:41 danicuki: The JAM0 channel was created during the Bangkok gathering. Nothing there is private, just we didn't wanted to spam the main JAM chat with conferences and meetups organization details. The meeting video will be shared with all implementors. Anyone is welcome to join. About the twitter thread in particular, fill free to send me your blurb and X handle, and I add it to the thread.

2025-01-14 19:43 danicuki: Summary of the Meeting Outcomes: 1/ 💼 Meeting Frequency: We agreed on holding monthly meetings to synchronize our efforts and tackle obstacles together. This will ensure that everyone stays updated on progress and challenges. 2/ 🎥 Recording Sessions: The team discussed recording our calls for transparency and sharing knowledge. These recordings will be made available privately and potentially shared publicly for broader community engagement. 3/ 🗓️ Upcoming Meetup: A second JAM0 meetup is being planned in Lisbon, likely around May, coinciding with Ethereum Lisbon. This event will focus on collaboration and knowledge sharing among JAM implementers. Waiting for gav availability to define dates. 4/ 🛠️ Community Test Vectors: We emphasized the importance of developing community test vectors to streamline our testing processes. This will help ensure compatibility and foster collaboration among different implementations. 5/ 📈 Milestones Discussion: Teams shared progress on their individual milestones, with special attention on the transition from the first milestone to the more complex second milestone, which involves networking and consensus mechanisms. 6/ 🌍 Networking Implementation: There’s a strong interest in collaborating on networking implementations across teams, ensuring that different JAM clients can communicate effectively. 7/ 🤝 Overall, the meeting reinforced our commitment to collaboration, transparency, and continuous improvement within the JAM community. Together, we aim to drive innovation and success in our projects!

2025-01-15 02:51 jay_ztc: Can someone invite me to the JAM0 channel please? Thanks!

2025-01-15 11:21 gav: I've seen a couple of proposals for JAM node RPCs: - https://docs.jamcha.in/basics/rpc - https://github.com/open-web3-stack/jam-rpc-spec?tab=readme-ov-file

2025-01-15 11:22 gav: The first thing I'd like to say is _Death to RPCs_! 🫣

2025-01-15 11:22 gav: In general, all JAM interaction should happen, as with smoldot, through an in-process (light) node. (edited)

2025-01-15 11:23 gav: However, I accept it's probably helpful to have some sort of RPC defined either for local interactions or development/debugging.

2025-01-15 11:23 gav: So, a slight critique of these two.

2025-01-15 11:24 gav: The first suffers from jam_getServiceStorage and, probably, jam_getState.

2025-01-15 11:24 gav: The jam_getServiceStorage RPC call, and probably the other also, are not generally implementable. (edited)

2025-01-15 11:26 gav: The protocol doesn't track inserted keys; the node would have to. This requires specialised logic and a potentially very large shadow database and removes the possibility of light-client implementations and fast-syncing. Given that block-execution will likely take a very large portion of the 6 second block time, full-syncing a chain of any significance is going to be practically impossible, doubly so if you're also having to retain a database of all inserted keys as well as everything else. (edited)

2025-01-15 11:27 gav: Otherwise the first proposal looks pretty sensible.

2025-01-15 11:29 gav: The second proposal has typos (never a good sign) and is not especially clear in meaning. In particular, state_getKeys and state_getStorage are not 100% clear on the meaning of the term "keys". If the meaning is the key as it appears in the JAM chain Merkle tree, then the calls are fine and make sense. If the meaning is the key as appearing in the read/write host-calls, then not. (edited)

2025-01-15 13:36 danicuki: JAM Community Call #1 video is live: https://www.youtube.com/watch?v=ghjvdHzA1P0

2025-01-15 21:26 tomusdrw: Are there any estimates when jam-sdk supporting 0.5.4 opcodes will be released? Or maybe there is any way to contribute to make this happen? :)

2025-01-15 21:26 gav: Tomorrow.

2025-01-15 21:26 gav: Actually…

2025-01-15 21:27 gav: Are they not already in place in the latest release from a few days ago?

2025-01-15 21:27 gav: If not then I guess it’ll depend on @jan:parity.io

2025-01-15 21:33 tomusdrw: let me double check, but afaict last release was 7 days ago and last time I checked it was producing pre-0.5.4 opcodes

2025-01-15 21:40 tomusdrw: yeah, jam-pvm-build=0.1.9 seems to be producing a bunch of jumps initially and then a bunch of 113 instructions which used to be STORE_IND_U64 and now is invalid.

2025-01-16 05:41 jan: We will cut a release today.

2025-01-16 10:07 danicuki:

2025-01-16 11:33 gav: polkajam-sdk v0.1.12 is released on crates.io https://docs.rs/jam-pvm-common/latest/jam\_pvm\_common/index.html (edited)

2025-01-17 18:48 danicuki: I've created an open source repo with some JAM media assets. It might be helpful to use on presentations, social media posts, etc. Contributions are welcome: https://github.com/jamixir/jam-media

2025-01-17 19:14 jaymansfield: > <@danicuki:matrix.org> I've created an open source repo with some JAM media assets. It might be helpful to use on presentations, social media posts, etc. Contributions are welcome: https://github.com/jamixir/jam-media Thanks for this. Definitely useful.

2025-01-19 17:41 celadari: Hi everyone, I have a question regarding the Safrole test vectors. Looking at this commit: 🔗 Commit 2d71c3f I noticed that the ring root values were changed, even for test vectors without offenders. ❓ Can someone clarify what changed at that point in the computation of the ring root? At the moment, we are using the following code (this part is from primitives, so I can share it here in the thread)

let ring_size = public_keys.len();
    let ring_ctx = ring_context(ring_size, pcs_params_file_path)?;
    let padding_point = Public::from(ring_ctx.padding_point());

    let mut keys: Vec<Public<BandersnatchSha512Ell2>> = Vec::new();
    for key in public_keys {
        let mut cursor = Cursor::new(&key);
        let pub_key = Public::<BandersnatchSha512Ell2>::deserialize_compressed(&mut cursor)
            .unwrap_or_else(|_| padding_point);
        keys.push(pub_key);
    }


    let pts: Vec<_> = keys.iter().map(|pk| pk.0).collect();
    let verifier_key = ring_ctx.verifier_key(&pts);
    let commitment = verifier_key.commitment();

    let mut buf = Vec::new();
    commitment
        .serialize_compressed(&mut buf)
        .map_err(|_| VRFResult::SerializationError)?;

    Ok(buf)

let ring_set: Vec<Public> = bandersnatch_keys;
let verifier = Verifier::new(ring_set);
verifier.commitment.serialize_compressed(&mut proof[..]).unwrap();
return proof;

let padding_point = Public::from(ring_ctx.padding_point());

let mut keys: Vec<Public<BandersnatchSha512Ell2>> = Vec::new();
for key in public_keys {
        let mut cursor = Cursor::new(&key);
        let pub_key = Public::<BandersnatchSha512Ell2>::deserialize_compressed(&mut cursor)
            .unwrap_or_else(|_| padding_point);
        keys.push(pub_key);
}

let padding_point = Public::from(ring_ctx.padding_point());
let public_keys: [u8; 32] = bandersnatch_keys;
let mut ring_set: Vec<Public> = Vec::new();

for key in public_keys.iter() {
        let point = Public::deserialize_compressed(key)
            .unwrap_or_else(|_| padding_point);
        ring_set.push(point);
}
let verifier = Verifier::new(ring_set);
...

> let padding_point = Public::from(ring_ctx.padding_point());
> 
> let mut keys: Vec<Public<BandersnatchSha512Ell2>> = Vec::new();
> for key in public_keys {
>         let mut cursor = Cursor::new(&key);
>         let pub_key = Public::<BandersnatchSha512Ell2>::deserialize_compressed(&mut cursor)
>             .unwrap_or_else(|_| padding_point);
>         keys.push(pub_key);
> }
> 

2025-01-19 19:10 danicuki: Some people noticed that the Polkadot logo used for JAM/Polkadot mix is an old version of Polkadot logo. gav Should we update it using the new one?

2025-01-19 19:12 piotrzwolinski: Hi! We have finalized the first "official release" of the Gray Paper Reader (https://graypaper.fluffylabs.dev), which most of you probably know. We believe this tool will serve as a living documentation resource even beyond the JAM Contest, supporting ongoing development and understanding of JAM. To keep the Reader freely available and open-source, we’re **submitting a proposal for retroactive funding** to the Polkadot Treasury. We’ve started a discussion on Polkassembly to share details about the proposal and gather feedback: https://polkadot.polkassembly.io/post/2722. Your thoughts and questions are welcome!

2025-01-19 19:15 boymaas: Great work! ❤️ I’ve been using it with pleasure much appreciated.

2025-01-20 22:19 decentration: can i confirm which hashing algorithm we are using for mmr peak hashes? im assuming sha2-256 by default.

2025-01-20 22:20 dave: MMR uses Keccak, most other things use Blake2b

2025-01-22 15:48 charliewinston14: Having some difficulty finding an erasure coding library that could adhere to the GP specs. Was previously using a branch of paritytech's erasure-coding repo (branch: cheme/test_vecs) but the PR has been closed as it was non conforment. What library are the other teams using for this?

2025-01-22 17:28 decentration: for M1, are there any conformance test vectors that we don't need to do until M2, in the davxy/jam-test-vectors repo? or shall we pass every (tiny) test from within assurances, authorizations, disputes, history, preimages, reports, safrole, statistics...? (edited)

2025-01-22 17:38 dakkk: Take a look on this issue: https://github.com/w3f/jamtestvectors/issues/21 I have doubt about erasure coding, since I'm not sure it is required for M1 (edited)

2025-01-22 18:01 gav: Edit: M2 is focussed on block authoring whereas M1 on block execution. The erasure coding, for example, will not be needed for M1 as it is not used in block execution. (edited)

2025-01-22 18:03 dakkk: From what I see on the website (https://jam.web3.foundation/), M2 is focused on block authoring, 3 and 4 are focused on performances

2025-01-23 08:41 gav: > <@dakkk:matrix.org> From what I see on the website (https://jam.web3.foundation/), M2 is focused on block authoring, 3 and 4 are focused on performances Yes indeed. Most test vectors will end up being useful for M1 but some are not relevant (such as erasure coding). In any case the actual conformance tests will be done privately and using a random fuzzer by the W3F to avoid the possibility of teams gaming their implementations to any blind spots in the published vectors.

2025-01-23 08:43 gav: The test vectors are there as an aid only. Don’t rely on them to tell you what you should and should not implement for M1 or M2. There is no short-cut to total comprehension of the Gray Paper. (edited)

2025-01-23 08:44 xlchen: is the confirmation test fuzzer going to be open sourced? so that we don’t need to invest too much time to integrate other fuzzers

2025-01-23 08:50 gav: The interface will be.

2025-01-23 08:51 gav: But probably not the block-generation aspects.

2025-01-23 08:51 gav: That would defeat the purpose.

2025-01-23 08:58 xlchen: ok so it will be more or less the acm contest style? we submit, get some vague error message if failed, and repeat until pass?

2025-01-23 09:07 gav: Well, given that the W3F's throughput is not infinite, teams relying on this "grind" strategy will likely not get anywhere fast.

2025-01-23 09:08 gav: For failed entries we will try to add a test vector to the public repository demonstrating how the candidate failed (by giving what it should have done). This may not always be possible, but we'll do our best. (edited)

2025-01-24 12:05 jan: I have just updated the PVM test vectors here; grab them fresh from the oven here: https://github.com/w3f/jamtestvectors/pull/3 This is the biggest update yet; changes: * The tests now target 64-bit PVM, in alignment with GP 0.5.4. (Thanks to Ivan Subotic for helping out with these!) * The trap exit status is now called panic to align with the GP. * Added new exit status: page-fault; tests which previously panicked when accessing unpaged memory now generate page faults. * Added 106 new tests from the RISC-V test suite transpiled into PVM. All of the tests starting with riscv_ are ported from an external RISC-V test suite, hence they're named based on which original RISC-V instructions they were testing. I've just transpiled them to PVM (since PVM is derived from RISC-V) and left them mostly unmodified. As always, please do let me know if there are any issues with these test vectors.

2025-01-24 12:06 boymaas: Nice! Thank you Jan Bujak

2025-01-24 23:07 jaymansfield: > <@jan:parity.io> I have just updated the PVM test vectors here; grab them fresh from the oven here: https://github.com/w3f/jamtestvectors/pull/3 > > This is the biggest update yet; changes: > * The tests now target 64-bit PVM, in alignment with GP 0.5.4. (Thanks to Ivan Subotic for helping out with these!) > * The trap exit status is now called panic to align with the GP. > * Added new exit status: page-fault; tests which previously panicked when accessing unpaged memory now generate page faults. > * Added 106 new tests from the RISC-V test suite transpiled into PVM. > > All of the tests starting with riscv_ are ported from an external RISC-V test suite, hence they're named based on which original RISC-V instructions they were testing. I've just transpiled them to PVM (since PVM is derived from RISC-V) and left them mostly unmodified. > > As always, please do let me know if there are any issues with these test vectors. Are the instruction codes for the risc-v tests aligned with the GP?

2025-01-25 09:25 gav: Given the above message, I’d say the answer is “yes”

2025-01-25 12:13 luke_fishman: question about import segments: as a guarantor i receive a work package that includes a sequence of imported data segments I need to import and reconstruct them, by fetching 342 shards. looking here(https://github.com/zdave-parity/jam-np/blob/main/simple.md#ce-139140-segment-shard-request) i see that in order to fetch a shard, i need to use the erasure root is this a mistake in the hackamd document? should it be fetched using the segment-root?

2025-01-26 13:42 decentration:

2025-01-26 14:18 gav: It should be unambiguous already.

2025-01-26 14:19 gav: It won’t be as nearly as easy as M1’s block-execution logic because it isn’t a simple closed-form spec, but more of a constraint-optimisation problem.

2025-01-26 14:20 gav: That’s just the reality of block authoring as opposed to block execution.

2025-01-26 15:57 luke_jamixir: OK, maybe it's a silly or too open ended questions, but could you shed some light please on 1.Why is the justification for a import segment is either a segment root hash OR a work package hash 2. Why is erasure root used to fetch segments and not the segment root? 3.being given a wp from a builder, what is the process to find its erasure root? (seems that in order to know erasure root I need to know the imports and in order to do that I need to know the erasure root) (edited)

2025-01-26 18:14 luke_jamixir: So the WP that exported it, do I get it's erasure root from the corrodponding WR (on chain, accumulated already) or?

2025-01-26 18:54 dave: Yes, you will need to get it from the WR. The assumption is that nodes will maintain a database mapping SR to ER, which they will do by monitoring WRs that are included on chain

2025-01-26 19:33 tomusdrw: FYI: I've built a web app to encode/decode JAM objects: https://papi.fluffylabs.dev it's a fork of an amazing papi-console, so most of the credit should go there - I just plugged in the JAM codec. Hope you'll find it useful. (edited)

2025-01-27 00:51 luke_jamixir: > <@dave:parity.io> Yes, you will need to get it from the WR. The assumption is that nodes will maintain a database mapping SR to ER, which they will do by monitoring WRs that are included on chain Thank you @dave:parity.io

2025-01-27 08:59 gav: Just a friendly reminder on clean-room requirements: don’t look at other implementation’s code (and that includes PolkaVM). Doing this will compromise you and your implementation. It’s not hard to see where code is copy-pasted (and use of the world’s most powerful plagiarism device, AI, is no different). The rules are clear: if you submit a code base with content determined to come from anywhere other than official specs, you’ll likely be disqualified.

2025-01-27 10:18 jan: To expand on what Gav wrote, in the context of PolkaVM (since that's a major piece of Parity's node implementation that is already open source and people may be tempted to look at) this should be obvious but let me explicitly state it - copying _any_ of these is not acceptable according to the rules (this list is _not_ exhaustive; these are just examples): - PVM program blob deserialization - instruction parsing - interpreter implementation of what each instruction does - exact interpreter architecture - machine code which the recompiler generates for each instruction - outer PVM memory map builder It's better to err on the side of caution and not look at PolkaVM at all, but there are some thing I would find acceptable to look at/copy/contribute to as long as it's declared up front that you've looked at these things (note that this is not "official" and I'm only speaking in a purely personal capacity here, but I will defend anyone who does any of these): - human readable disassembly pseudo code syntax (that is, the way instructions are printed out as seen in the TESTCASES.md in my PVM test vectors; there's no official way to disassemble PVM programs besides what they're named in the GP, but feel free to copy my pseudo code syntax if you like it), - bindings to native Linux APIs (polkavm-linux-raw crate) and x86/AMD64 assembler (polkavm-assembler crate) - these are just general purpose libraries that only coincidentally live in the PolkaVM repository for convenience; the goal here is for people to become experts in the JAM protocol, and not become experts in how to write high performance x86 assemblers (which you need for a very fast recompiler, but by itself is not specific to the JAM protocol), - high level API as shown on https://docs.rs/polkavm (this is just an interface, not an implementation; even PolkaVM's API was/is partially based on wasmtime's API), - high level techniques for implementing a fast parser or a recompiler, since it's unrealistic to expect 34 independent teams to rediscover these from scratch, especially since those are useful/used when implementing _any_ VM, not just PVM (but I emphasize the _high level_ part - just copying PolkaVM's code here is not acceptable; if there's interest eventually I'll host a workshop where I'll explain some of these independent of any particular implementation), - the toolchain to build PVM programs (polkavm-linker crate, polkavm-derive, polkatool, etc.) - this takes care of translating RISC-V code to PVM; this runs offline and is not part of the GP, - the format of the .polkavm blob - the on-disk format of what the toolchain emits by default; this is not part of the GP and will never be (note that this only applies to the parts of the blob that are specific to the .polkavm blob and are _not_ specced by the GP!), - the format of the debug info inside of .polkavm blob (is not and will never be specced by the GP; this is needed to be able to debug PVM programs and it's useful for production-level VMs to support some form of debug info) In general please use some common sense, and when in doubt please first ask. (edited)

2025-01-27 13:20 luke_fishman: > if there's interest eventually I'll host a workshop where I'll explain some of these independent of any particular implementation just stating my interest. i would be very happy to join any workshop you may host on the topic

2025-01-27 13:21 boymaas: Good point Luke | Jamixir Would be very happy as well!

2025-01-27 17:10 yu2c: Hello, I have a question about PVM implementation. Since we chose a non-Rust language to implement the JAM protocol, I feel that using a higher-level language makes PVM implementation more challenging. Are there any language restrictions for PVM implementation?

2025-01-27 17:11 dakkk: what do you mean with "language restrictions"?

2025-01-27 17:24 yu2c: If we choose Golang to implement the JAM protocol, does that mean our PVM implementation is also restricted to Golang? Or we can implement our PVM by Rust?

2025-01-27 17:58 gav: You are free to mix-and-match languages from the same language set.

2025-01-27 18:01 gav: Things get complicated with mixed-language implementations from two (or more) different language sets. For this, the Fellowship and W3F will evaluate the footprint of each language in the implementation and a fractional prize may be taken from the according sets. This fraction will be entirely at the judges discretion. (edited)

2025-01-27 18:02 gav: To avoid being at the mercy of such an arbitrary decision I’d suggest teams stick to a single language set.

2025-01-27 18:19 yu2c: Clearly understood! But I have another question: if I use Rust for some low-level implementation of the PVM and construct the PVM's high-level part in Golang by leveraging FFI (Foreign Function Interface) from Rust to Go, would this operation potentially go beyond the single language set? Let me broaden the question: for third-party libraries (e.g., VRF, erasure coding, ...) allowed under the JAM Prize rules, would using FFI to bridge two language sets also go beyond the single language restriction?

2025-01-27 18:36 yu2c: I understand your answer regarding FFI here, but I’d like to focus on the issue of language sets: would using FFI between two different language sets be considered mixed-language set implementation under the rules?

2025-01-27 18:49 gav: Language is unimportant and irrelevant for the third-party utility libraries for those portions of the logic not covered by the prize (crypto, erasure-coding, networking, ...). (edited)

2025-01-27 18:50 gav: Language only matters for that code written and submitted against the JAM Prize (commonly known as a "JAM Implementation"). (edited)

2025-01-27 18:52 gav: > if I use Rust for some low-level implementation of the PVM and construct the PVM's high-level part in Golang by leveraging FFI (Foreign Function Interface) from Rust to Go, would this operation potentially go beyond the single language set? Yes. It would most certainly and very obviously go beyond a single language set: your implementation is using two languages! (edited)

2025-01-27 18:59 yu2c: Got it, thanks for the clarification!

2025-01-27 19:58 tomusdrw: Correct me if my take on this is wrong, but my understanding is that the interpreter preformance is pretty irrelevant anyway, since to reach anywhere near the rust implementation (for m3/m4) it will require some transpilation of PVM code to native code (compilation). I have a pretty limited knowledge on this, but afaict compilation involves: a) bringing in something that can turn assembler into native code (most likely llvm) and just producing native code for the hardware you are running on. For languages with runtimes it will require FFI anyway and the language-of-choice can only orchestrate that process.

2025-01-28 03:48 clearloop: curious about if it is possible to hit m3 with an interpreter (edited)

2025-01-28 03:50 gav: > <@clearloop:matrix.org> curious about if it is possible to hit m3 with an interpreter Highly unlikely.

2025-01-28 07:05 dakkk: Anyway there are various techniques to improve performance even on interpreted languages; for instance, for me using python I can rewrite critical sections of jampy using cython (which is python code translated to C and compiled)

2025-01-28 08:54 jan: Take a look at benchmarks here: https://github.com/paritytech/polkavm/blob/master/BENCHMARKS.md#oneshot-execution-for-pinky Compare "PolkaVM (64-bit, recompiler)" vs "PolkaVM (64-bit, interpreter)". My interpreter can still be optimized, but even if you look at state of art optimized-to-the-bone interpreters (e.g. wasm3) there's still an order of magnitude performance difference vs the recompiler. So essentially I expect the M3 performance threshold will most likely be on the level of "a working but not necessarily optimal recompiler with okay compilation speed" and M4 to be "an optimized recompiler with high recompilation speed".

2025-01-28 18:03 sourabhniyogi: The "JAM requires linear time O(n) recompilation" has been an non-explicit (but deepseekly awesome) requirement it appears, for M3 -- check? Can you explain what teams would be expected to do to achieve this O(n) linear compilation time, and how we could get started on this process in the next 2 months? It would appear that once we go through the above transformation, the bulk of our Appendix A PVM implementations are then expendable training wheels, save the infamous ecalli (and thus as tomusdrw says, anyone's interpreter performance is in fact _irrelevant_), check? If this is correct, we should like to understand how this ecalli / host function call process works in a post-recompilation era, soon? (edited)

2025-01-28 19:15 gav: > as tomusdrw says, anyone's interpreter performance is in fact irrelevant Almost certainly correct.

2025-01-28 19:17 gav: Jan Bujak is already planning on giving talks about how to achieve a high performance O(n) recompiler. Any team interested in delivering M3 or M4 would be well-advised to pay very close attention.

2025-01-28 19:18 gav: It won't just be about the recompiler though. Much performance will come with optimised databases, Merklisation and networking behaviour. M4's performance tests (which have yet to be written) will be stringent.

2025-01-28 19:19 gav: M3 is intended to be somewhat less stringent but still requiring a recompiler and sensible implementations for all of the above. Obviously, some languages will need to be less optimised than others in order to get to M3.

2025-01-28 23:03 decentration: do you mean don't use AI as a tool for plagiarism, or that AI is itself plagiarism? it is a very clear rule not to look at anything related to JAM implementation except for the GP, these two channels, the JAM videos and the conformance tests. But what is now less clear are AI tools use. i thought it was clear when you mentioned that AI should not be used to plagiarise someone else's work, in [June 2024](vector://vector/webapp/#/room/!wBOJlzaOULZOALhaRh:polkadot.io/$aA1-kjwr08g74hQ46SpIIBajbxSFsZl_hg5RfRcWvpQ), this makes sense and is an understandable concern, hence why commits must show a timestamped path of one's own coding journey. But it has never been mentioned that it cannot be used at all. Can you confirm? Because the "Explain" button in the Fluffy Labs GP reader, signals that it can be used, but as an obvious caveat not to be used as a tool to rearticulate someone else's work. (edited)

2025-01-28 23:16 jimboj21: Hello, I have a question regarding the MMR super peak function (GP E.10). I thought I have an understanding of how this works, but am not getting the correct output on the reporting vectors test case for this. Lets say the input to the function is: [a, b, nil, c] I would expect the following behavior for this example: - empty removed, giving [a, b, c] - msg = $peak + a + b - msg2 = $peak + H(msg) + c - super_peak = H(msg2) Does this seem like the correct flow (i reduced the details in the example, but can elaborate on steps if needed)? I'm not understanding where I am off in my logic here but like I said I am not outputting the correct value (unless its a bug elsewhere). Any input is appreciated, thanks!

2025-01-29 00:33 gav: > <@decentration:matrix.org> do you mean don't use AI as a tool for plagiarism, or that AI is itself plagiarism? > > it is a very clear rule not to look at anything related to JAM implementation except for the GP, these two channels, the JAM videos and the conformance tests. > > But what is now less clear are AI tools use. i thought it was clear when you mentioned that AI should not be used to plagiarise someone else's work, in [June 2024](vector://vector/webapp/#/room/!wBOJlzaOULZOALhaRh:polkadot.io/$aA1-kjwr08g74hQ46SpIIBajbxSFsZl_hg5RfRcWvpQ), this makes sense and is an understandable concern, hence why commits must show a timestamped path of one's own coding journey. > > But it has never been mentioned that it cannot be used at all. Can you confirm? Because the "Explain" button in the Fluffy Labs GP reader, signals that it can be used, but as an obvious caveat not to be used as a tool to rearticulate someone else's work. It is a slightly impetuous and controversial remark but my point was that current AI tools are fundamentally plagiaristic.

2025-01-29 00:34 gav: Generally they are shallow reformulation of the work of others and usually devoid of original material.

2025-01-29 00:36 gav: We can’t really enforce the non-use of AI more than we can enforce not looking at, say, stackexchange. But if we see obvious similarities in two code bases we will assume at least one is plagiaristic.

2025-01-29 00:37 gav: So beware of this if using ChatGPT to write your algorithms for you.

2025-01-29 05:05 stanleyli: Hello everyone, I just created a PR to jamtestvector, providing test vectors generated based on the definitions from the Gray Paper. https://github.com/w3f/jamtestvectors/pull/35 I hope those interested can give them a try, and I look forward to contributing to the community

2025-01-29 07:49 dakkk:

2025-01-29 07:50 dakkk: We talked about it in this channel in September, if I remember correctly

2025-02-03 11:08 emielsebastiaan: I found an error in testvectors for the rem_s_64 instruction: https://github.com/w3f/jamtestvectors/pull/3#issuecomment-2630619549

2025-02-03 12:35 decentration: do we have a tiny const for L (re: 11.34)? In GP (I.4) Values it's:

L = 14, 400: The maximum age in timeslots of the lookup anchor.

(edited)

2025-02-03 12:47 oliver.tale-yazdi: I dont think there was a value proposed for this yet, otherwise i will add it to https://docs.jamcha.in/basics/chain-spec/tiny We can probably calculate backwards from the toaster full spec (modulo any contraints)?

2025-02-03 18:11 gav: IMG_4368.jpeg

2025-02-03 18:11 gav: IMG_4367.jpeg

2025-02-03 18:11 gav: IMG_4366.jpeg

2025-02-03 18:12 gav: IMG_4365.jpeg

2025-02-03 18:12 gav: IMG_4364.jpeg

2025-02-03 18:12 gav: IMG_4363.jpeg

2025-02-03 18:12 gav: IMG_4362.jpeg

2025-02-03 18:12 gav: IMG_4361.jpeg

2025-02-03 18:12 gav: IMG_4360.jpeg

2025-02-03 18:13 gav: Some new pics on the latest state of the Toaster.

2025-02-03 18:13 gav: We now have around half of the machines delivered for phase 1 (41 of them), and should have the remaining 45 in the next 2-3 weeks. (edited)

2025-02-03 18:13 gav: The first 6 are being configured now in a temporary server room, the others will be installed and configured in the dedicated space once that's ready.

2025-02-03 18:14 gav: They've been installing some serious heat ducts to make sure that the jacuzzi gets toasty.

2025-02-03 18:18 gav: Once phase 1 is built out and going strong we can start to think about phase two, which will see an additional 85 machines for a total of 171 machines. That will take us to our goal of 16 AMD TR cores, each with 1MB L2 cache, for each of the 1023 nodes. (edited)

2025-02-04 20:09 decentration: > <@gav:polkadot.io> They've been installing some serious heat ducts to make sure that the jacuzzi gets toasty. nice use case 👌

2025-02-05 13:54 decentration: Do we need to enforce a particular order for the Set of Ed25519 keys, Reporters (R)? from gp 11.26 i believe we don't, but just checking if we need it to go beyond and conform to the order of test vector exactly. e.g. reports_with_dependencies-1.json

2025-02-05 14:09 gav: Do you have a GP link?

2025-02-05 14:10 decentration: https://graypaper.fluffylabs.dev/#/4bb8fd2/15b80015b800 i ordered from ascending validator\_index but seems test vector is slightly different (edited)

2025-02-05 14:15 gav: 11.26 only provides constraints for the items.

2025-02-05 14:16 gav: 11.25 constrains the ordering of the items.

2025-02-05 14:16 gav: (by v).

2025-02-05 14:26 decentration: ok ty, yes i'm ordering by v \[0,1,2,3,4,5\] but seems test vector is doing something different, i.e. \[1,0,4,5,3,2\] in reports\_with\_dependendencies-1.json, will pr in the repo. (edited)

2025-02-06 13:23 clearloop: curious how many teams are now following jamnp? when will there be a stable spec for the network (p2p) implementation? will there be an official transaction sender client for testing & debugging? (edited)

2025-02-07 14:45 ycc3741: Does implementing BLS Signatures require a PoPs version, or is a non-PoPs version sufficient?

2025-02-07 14:45 ycc3741: Refer to https://eprint.iacr.org/2022/1611.pdf.

2025-02-07 16:09 wind.sixwings: Is there a bls signature length defined in GP? We need to ensure that the bls signature length is consistent

2025-02-08 23:22 rustybot: > <@decentration:matrix.org> this is helpful thread, thanks. so to pass conformance test use "node" instead of "peak" as prefix. Vectors were regenerated to use "peak". Have you tried to pull the last rev?

2025-02-10 11:38 sourabhniyogi: Here is our open source implementation of erasure coding in Go (Appendix H) with a matching erasure coding test vector: https://github.com/jam-duna/jamtestnet/tree/main/erasurecoding We would love others providing bug reports on this -- once its believed to be compliant by 2+ teams, we imagine other language implementations can match.

2025-02-10 19:10 lucy_coulden: Attended the first JAM Tour session at Cambridge University which was cracking! I have a couple of questions… - how do people apply to be involved as a builder? - I believe there are 35 official teams. Is there a list of these teams and where they are based? We are doing a big push in the US and exploring a physical footprint which may be called a JAM space and where people could go to build and be together with others in the Polkadot ecosystem.

2025-02-10 19:12 xlchen: you can find registered teams here https://graypaper.com/clients/

2025-02-10 21:32 ascriv: +1 for jam hacker spaces post m5

2025-02-10 21:56 sourabhniyogi: image.png

2025-02-10 21:56 sourabhniyogi: Some JAM implementers are getting together in a "JAM0 Meetup #2" call in the next 24 hours. If you are implementing JAM, please add this to your calendar and DM me (@sourabhniyogi) and I'll you to the "JAM0" Element channel where a meeting link will be dropped.

2025-02-10 21:59 jaymansfield: > <@sourabhniyogi:matrix.org> Some JAM implementers are getting together in a "JAM0 Meetup #2" call in the next 24 hours. If you are implementing JAM, please add this to your calendar and DM me (@sourabhniyogi) and I'll you to the "JAM0" Element channel where a meeting link will be dropped. > > Looking forward to meeting the other teams tomorrow!

2025-02-11 01:25 lucy_coulden: Thanks Bryan! Is there anything that lists where in the world the teams are based? (edited)

2025-02-11 01:27 lucy_coulden:

2025-02-11 04:41 xlchen: I am not aware of any public list with team location. Some of the teams are remote / global also.

2025-02-11 12:25 alice_und_bob: can services pull in memory pages from other services? I am asking because I wonder how separated different applications in JAM are across the service boundary. E.g. can a corechains chain's logic be based on the state of a coreplay agent? (edited)