How this was built
Methodology
Which dataset supplied which field, what each one was forbidden to supply, what was thrown out and on what rule, and the numbers this site could not source and therefore did not print.
Last reviewed 6 September 2026
The principle
A page on this site exists because a row exists in a dataset. Nothing is generated from a template, nothing is written to fill a gap, and no figure is recalled rather than sourced. Where a value cannot be traced to a document, the field is absent and the page says the field is absent.
That rule has a visible cost. 209 pickles are documented here and only 23 of them carry a published fermentation parameter. The other 186 have a name, a region, a substrate and a row of blanks. That ratio is not a gap in this site; it is the state of the published record for traditional fermented vegetables, and the FAO bulletin says so itself in its closing chapter: most of this knowledge has not been documented and is in danger of being lost.
The datasets
| Dataset | What it supplies | Retrieved | Rows |
|---|---|---|---|
| FAO Agricultural Services Bulletin 134 Battcock, M. and Azam-Ali, S. (1998) Fermented fruits and vegetables: A global perspective. FAO Agricultural Services Bulletin No. 134. Food and Agriculture Organization of the United Nations, Rome. | Salt, brine, temperature, duration, pH, organisms and process description for 21 named processes; 22 further products named without parameters; a published index of 82 named products by region (its Table 2.1, from Campbell-Platt 1987). | 2026-09-06 | 87 parameters |
| USDA, via the National Center for Home Food Preservation National Center for Home Food Preservation, University of Georgia. Adapted from the Complete Guide to Home Canning, Agriculture Information Bulletin No. 539, United States Department of Agriculture, revised 2015. | Temperature bands with a stated consequence at each end, durations, salt by volume, the defects table, and the statements quoted on the safety page. | 2026-09-06 | 17 parameters |
| eCFR: 21 CFR 114 and 108 | The pH 4.6 threshold and the definitions around it, unedited. Issue date 2026-08-31, current as of 2026-09-03. | 2026-09-06 | 17 sections |
| USDA FoodData Central | Composition per 100 g, sodium above all. Foundation, SR Legacy and Survey only; the Branded database is not read. | 2026-09-06 | 347 foods |
| Tamang, J.P., Watanabe, K. and Holzapfel, W.H. (2016) PMC4805592, CC BY 4.0 | Product name, substrate, country and organisms. No numbers. | 2026-09-06 | 38 rows |
| Swain, M.R., Anandharaj, M., Ray, R.C. and Parveen Rani, R. (2014) PMC4058509, CC BY 3.0 | Product name, substrate, country and organisms. No numbers. | 2026-09-06 | 47 rows |
| English Wikipedia and Wikidata Wikipedia text CC BY-SA 4.0; Wikidata claims CC0 1.0 | The index of names, plus region, substrate and native-script label. Forbidden to supply any number anywhere on this site. | 2026-09-06 | 296 candidates |
| NCBI Taxonomy | Current scientific name, taxonomy id, rank and lineage for each organism named. | 2026-09-06 | 66 of 76 |
The methods, extracted separately
The FAO bulletin and the USDA guides are read twice by two scripts that do not share code. extract_fao.py and extract_nchfp.py answer "what number does the source state". extract_procedures.py answers "what does the source say to do", and is what the recipes are built from: 23 procedures carrying 124 steps, last built 2026-09-06.
Two rules govern it. Every step is a sentence lifted whole and verified character for character against the mirrored page, exactly as a parameter is; the step labels above those sentences are the only words on a recipe page this site wrote. And it copies no quantities at all. A figure lives in one dataset only, and a number with two homes is a number with two chances to be wrong and no way to tell which home was corrected.
Where a source records a method in four sentences, the recipe page is four steps long. The procedure file will not invent a step to make a page look complete, which is why the recipe index prints the step count next to every entry.
Which source may write which field
This is the rule the whole site rests on, and it is enforced in code rather than by intention. A salt percentage, a brine strength, a temperature, a duration, a terminal pH or an acidity may be written only by the FAO bulletin, the USDA guides or the Code of Federal Regulations. The build script asserts it and fails if a numeric field ever arrives carrying any other source. Wikipedia, Wikidata and the two review papers supply names, regions, substrates and organism lists, and nothing that has a unit.
The reason is not that encyclopaedias are unreliable in general. It is that a number here is an instruction somebody may follow with food, and the provenance of an instruction should be a document you can open, not a page that anyone may edit between one build and the next.
Why every parameter carries a sentence
Each parameter is stored with the sentence it was taken from, and the extraction scripts refuse to write anything if that sentence is not present, character for character, in the mirrored source. It runs on every rebuild. Transcription is where a reference site gets a number wrong, and the check costs nothing.
The sentence is also printed on the page, under the value. A reader who thinks a figure looks wrong can see immediately whether the site misread the source or the source says something surprising, which are different problems with different fixes.
The degree sign, and what it nearly cost
The 1998 FAO typesetting writes a degree sign as a superscript zero: 24<sup>0</sup>C. Strip the tags naively, as the first pass of the extraction did, and "21 to 24 C" becomes "210 to 240 C". Every temperature in the bulletin silently multiplies by ten, and an olive fermentation at 240 C is exactly the kind of figure that gets published on a reference site and repeated for years afterwards.
The converter now maps that markup to a degree sign before anything else happens, and the quote check would have caught the error anyway on the next run. It is recorded here because the class of bug matters more than the instance: a source that renders correctly in a browser can still be lying to a parser.
What was excluded
The candidate index came from the pickle and fermented-vegetable category tree on English Wikipedia, the A-Z body of "List of pickled foods", and the fermented fruit and vegetables section of "List of fermented foods". Those seeds also link to countries, plants, brands, museums and sandwiches, so the candidates go through a sieve. Every drop is recorded with a reason and the totals are published here, because an unpublished filter is indistinguishable from a hand-picked list.
| Rule | What it drops | Count |
|---|---|---|
| not-a-food | The article is a country, a region, a plant, a microbe, a trademark, a museum or a citation identifier. | 74 |
| residual-list | Named below in full: raw ingredients and abstractions that no mechanical rule catches. | 41 |
| animal-substrate | Fermented fish, cured meat and pickled eggs. All fermented, none of them a vegetable, and the parameter model here does not transfer to them. | 30 |
| overview-or-abstraction | List articles, cuisine overviews, and the generic 'Pickled radish' type pages that duplicate a substrate page here. | 24 |
| out-of-subject | Soy sauce, bean paste, fish sauce, cheese, kombucha and cassava staples. | 18 |
| dish-made-with | A dish that contains a pickle rather than a pickle: kimchi fried rice, pickle soup, a cheese and pickle sandwich. | 17 |
A candidate that a parameter source already names is never dropped. If the FAO bulletin gives something a numbered section, no filter here overrules that.
Seven fermented cassava staples from one of the review tables - gari, fufu, chikwangue and their relatives - were also dropped. They are lactic fermentations and they are not pickles, and a site that annexed them would be a directory of two subjects and an authority on neither.
The residual list, in full
41 articles reach the end of the sieve and are still not a named pickled preparation. There is no rule that catches them, so they are named here instead:
Airplane olive, Apple, Artichoke, Awaze, Bagoong, Balkan, Bean salad, Bird's eye chilli, Brinjal, Chinese mustard, Chinese pickles, Coconut water, Condiment, Cooking oil, Durian, East Asian cuisine, Friendship, Garum, Gherkin, Gimjang, History of Mesopotamia, Indian pickles, Japanese tea ceremony, Kapusta kiszona duszona, Lactic acid, Mango, Minnesota sushi, Mixed pickle, Nshima, Okazu, Philippine condiments, Pickled fruit, Pickling salt, Prune, Rice, Salt, Shelf life, Side dish, Sourness, Turnip, Yongfeng, Shuangfeng.
Organism names
The review tables abbreviate genera: Lb., Leuc., Ped., W., Ent., Lc. Those are expanded against a fixed map. A bare "L." is genuinely ambiguous in this literature - it is Lactobacillus in one row and Leuconostoc in the next - so it is expanded only where the species epithet appears elsewhere in the same corpus with its genus spelled out, and the resolution is learned from the data rather than assumed.
Each name is then resolved against NCBI Taxonomy. 38 of 76 carry a name that has since been reassigned, nearly all of them in the 2020 split of Lactobacillus. Both names are printed and the join is NCBI's. 10 did not resolve at all; most are typographical errors in the source papers, and this site prints them as published rather than correcting them, because deciding which organism an author meant is not a decision it is entitled to make.
A note on the lookup itself, since it is a trap: the NCBI E-utilities taxonomy index does not contain the superseded binomials. A search for Lactobacillus plantarum returns nothing at all, not a redirect. Since the renamed organisms are the entire point of the exercise, that is a total failure disguised as a clean run of empty results. The resolution goes through the datasets endpoint as well, and only accepts a suggestion when the term it reports matching is the name that was asked for.
Gaps left open
- Degrees salometer. The FAO bulletin gives four brine figures on that scale. This site could not source the definition of the scale to a published authority it was willing to cite, so those figures are printed as the bulletin printed them and are not converted. The factor is widely repeated and is probably right. Probably is not the standard a controlling number is held to here.
- A salt percentage from the USDA figures. The USDA gives salt by volume - three tablespoons per five pounds of cabbage. Converting that to a percentage by weight requires a bulk density, and the same guide warns that flake salt varies in density enough that it is not recommended for pickling. Publishing a derived percentage would be publishing an assumption as a measurement, on the one number this subject turns on.
- Terminal pH for almost everything. Two traditions here have a published terminal pH. Both are non-salted Himalayan ferments that happen to have been studied. For the rest, nobody has published one.
- Fermentation status for 35 entries. They are in the pickle categories and no source consulted says they ferment. They keep their pages and their pages say so.
Rebuilding this
The raw downloads are mirrored to a gitignored directory and the derived JSON is committed, so a build never touches the network and a rebuild is reproducible. The pipeline is these scripts, run in this order:
fetch_fao.sh, extract_fao.py, extract_nchfp.py, extract_cfr.py, extract_epmc.py, fetch_wikipedia.py, build_traditions.py, extract_fdc.py, extract_taxonomy.py, extract_procedures.py, resolve-amazon.mjs