BF: P1 format correctness - Nastran, Medit, Abaqus, Gmsh, PLY/STL, Tecplot - #47
Merged
Merged
Conversation
VARLOCATION was refused outright, so a zone with cell data could not be read at all. Parse the clause instead: a cell-centred variable's BLOCK run is E long, lands in element_attrs, and the writer emits it back under VARLOCATION=(...=CELLCENTERED), switching the zone to BLOCK packing. A clause that cell-centres a coordinate, indexes past the variable list, appears without VARIABLES, or asks for cell data under POINT packing is still refused. File and zone titles now survive in global_attrs under tecplot_title and tecplot_zone_title (meshio #1372), so a multi-part conversion no longer loses the part names. meshio #1368, #1372.
The .inp parser read *NODE and *ELEMENT and nothing else, and its element table held fifteen cards, so a deck from any real solver lost most of itself. Reading: *NSET / *ELSET become vertex_tags and element_tags, declared on the block or standalone, with or without GENERATE (#1527); several *NODE blocks accumulate and a repeated id restates that node rather than appending a stray vertex (#1528, #309); *INCLUDE is resolved against the including file's directory, refusing an absolute path, one that escapes that directory, and nesting past eight files (#1531); *SYSTEM shifts and rotates the node blocks that follow it (#1569); each *PART / *INSTANCE is read under its own node numbering, merged, and tagged by name (#1456). Element table: cards now resolve on their base name with the modifier suffixes stripped, so CPS8R reads as CPS8 (#1515) and C3D20RH as C3D20; the table covers the C3D, CPS, CPE, CPEG, CAX, S, SC, R3D, M3D, DC, COH, T and B families (#320, #1393). Writing: element_type= forces the card written for a polyxios type (#1299), tags are written back as *Nset / *Elset cards, and quadratic types are no longer refused - meshio reads the result (#1529).
Gmsh type 13, the 18-node prism, was skipped because its ordering relative to VTK was unsettled. It is settled here from VTK's own edge and face lists: the 15-node wedge's permutation plus the three quadrilateral face centres, whose last two swap because Gmsh's face table runs (0,1,4,3), (0,3,5,2), (1,2,5,4) where VTK runs (0,1,4,3), (1,2,5,4), (2,0,3,5). It reads as a biquadratic_quadratic_wedge and writes back. meshio reads that element straight through - the disagreement is recorded in a test rather than left to be discovered later. Type 14 stays skipped: VTK has no 14-node pyramid. $NodeData and $ElementData are read into vertex_attrs and element_attrs at any component count the file declares, scattered by the tag each row names so a partial field lands where it belongs, and written back the same way. A malformed or unmatched field costs that field alone. Regression tests kept for the issues polyxios was already immune to: a single-line mesh round trips (#1421), reading prints nothing (#1404), mixed cell types survive a write (#865, #524), and a flat mesh keeps its zero z (#1116). meshio #1517, #1281, #1421, #1404, #865, #524, #1116.
A Nastran card name does not say the element's order: CTETRA is a 4-node tetrahedron with four grid point fields and a 10-node one with ten. The reader kept the corner nodes and dropped the rest, so every quadratic element in a deck came back linear. The card table now lists the shapes each name can hold and the reader takes the largest that fits the grid points actually present, stopping at the first blank field - Nastran allows a mid-side point to be left out, and that card is a linear element, not a quadratic one with a hole. CPENTA's mid-side nodes are permuted into VTK order (Nastran runs the vertical edges before the top ring, VTK after); every other card already agrees. The table also covers the one-dimensional families that used to read as an empty mesh - CBAR, CBEAM, CROD, CONROD, CTUBE, CBUSH, CGAP, CBEND, CVISC - plus CSHEAR, CTRAX3/6, CTRIAX6 and the CQUADX cards. CONROD names a material where the others name a property, so its grid points sit one field earlier. A shell's ZOFFS lands in element_attrs["zoffs"] when a deck carries one and is written back (#1396). Every higher-order type now has a write mapping, so a quadratic mesh survives an export. meshio #1505, #1396.
Medit ASCII had no codec at all - '.mesh' meant MFEM and nothing else - so a bamg or INRIA deck read as an MFEM header error. The registry now lets a format share its own extension: a codec listing an extension in both EXTENSION(S) and SNIFF_EXTENSIONS is saying it shares rather than owns it, and the dispatcher that results takes its writer from whichever codec declares SNIFF_DEFAULT_WRITER. '.mesh' therefore resolves by content - MeshVersionFormatted here, MFEM mesh there - while a bare write(mesh, "out.mesh") still emits MFEM as it always has. Write Medit with fmt=".medit" or a '.medit' path. The new codec reads every entity section (#1511, #1508), tolerates what bamg writes - the count on the keyword's own line, no closing End (#1311) - and pads a 2-D file's z. References become element_attrs["ref"] and one element_tags["ref_<n>"] group per distinct value, so a region label survives into a .vtk as a named group (#1258); the binary codec now does the same, and also decodes the Edges, Prisms and Pyramids sections it used to step over. The higher-order sections are skipped with a warning naming them. libMeshb's own documentation fixes no node ordering for high-order elements and defers it to a companion *Ordering section, so reading one would mean guessing a permutation - a silently bent element is worse than a skipped one. meshio #1511, #1508, #1311, #1258.
PLY: a line element went out as a two-vertex face list, which any reader takes for a degenerate polygon rather than the line the mesh held. Lines now travel as 'element edge' with vertex1/vertex2, the spelling the spec gives them, and are read back from that as well as from the vertex_index1/2 and bare-integer-pair forms files in the wild use. Edges are appended after the faces so a per-face attribute keeps lining up with its faces, and an element index no vertex answers to is now refused rather than read into a mesh nothing can draw. STL: a binary facet's attribute word is where the tools that colour an STL put the colour - five bits per channel, top bit set to say the word holds one - and it was being discarded. Facets that claim a colour land in element_attrs["colors"] as RGB in 0..1, the ones that do not stay NaN so an uncoloured facet in a coloured file is not read as black, and a file where none claims one grows no attribute. Colours are written back on a binary write; an ASCII write reports that it dropped them, since ASCII STL has no field for one. Regression tests kept for #1470, which polyxios is immune to: the reader already merges coincident facet corners, so an STL of shared corners does not explode into a per-facet point cloud on the way to a .ply. meshio #1394, #1355, #1470.
WKT (#1382): the ISO/SQL-MM surface family was refused outright. TRIANGLE reads as one triangle, TIN as a set of them, POLYHEDRALSURFACE as a set of polygons, EMPTY included. A TIN patch that is not a triangle, or a TRIANGLE carrying an interior ring, is refused rather than read as something else. FLAC3D (#584): the plan expected this one to be immune; measuring said otherwise, at fourteen times the file size in peak memory. Coordinates, connectivity, offsets and types now accumulate into machine-number arrays rather than Python lists of boxed objects, which takes a 29 MB grid from 14x down to 5.6x. A regression test guards a loose bound. Netgen (#1369): 'facedescriptors' - which every second-order Netgen mesh carries - was read as a section nobody recognised. It is now a known section, stepped over by name and reporting how many records it dropped. Regression tests kept for the ones polyxios is already immune to: TetGen .ele attributes are parsed as float64 and keep their fractions (#1318), SU2 boundary markers survive a write and a Medit mesh converts to SU2 (#1429, #1419), and a Gmsh mesh carrying physical groups writes to DOLFIN XML with the elements that have no place there reported rather than erroring (#265). meshio #1382, #584, #1369, #1318, #1429, #1419, #265.
Nastran read CHEXA20 and CTRIAX6 straight through where their mid-side grid points do not sit where VTK wants them, bending every quadratic brick and axisymmetric triangle a deck carries. Both are permuted now, keyed by card rather than by element: CTRIA6 and CTRIAX6 hold the same six points and number them differently. CTRIAX6 also names a material where the shells name a property, ZOFFS is read from CTRIA6 and CQUAD8 as well, and a CBUSH grounding its second end is skipped rather than sinking the deck. Binary STL mixed the two facet-colour conventions: Magics' channel order with VisCAM's valid bit. They are told apart by the COLOR= record Magics writes in the header, and an integer colour column now counts to 255 rather than saturating at 31. Gmsh crashed on an element tag no number can be made of, and dropped a whole $ElementData field when one row named an element of a type it had skipped. The tag costs nothing but the row it names. Also: PLY stretches a face property over the edges that follow it and reports a short edge record as a codec error; Medit ASCII carries the file's Dimension back out; Abaqus matches set names without regard to case, reports an element id defined twice and refuses an element_type= card that cannot hold what it was given; Tecplot wraps a BLOCK run; the registry lets only an owner keep a shared extension's writes, and only one; FLAC3D hands back the vertices without copying them again.
The ASCII PLY reader walked the vertex, face and edge blocks in a fixed order rather than the order the header declares them, so a file naming `element edge` ahead of `element face` read each block as the other's - silently wrong geometry when the counts happened to fit, a bare IndexError when they did not. The blocks are now consumed in declared order, an element this codec has no place for costs its own records and nothing else, and a short record names the format instead of escaping as an IndexError. Resolving the property positions once rather than per record takes a 17 MB file from 0.46s to 0.39s. Abaqus: - An assembly keeps almost all of its sets outside the instance that numbers them, naming it with INSTANCE=. Those ids were resolved against a map cleared at *End Instance, so every such set was dropped as undefined. Each instance's numbering is now kept past its own end, and one that only places its part shares the part's. - A GENERATE range is two numbers with nothing bounding their distance: `1, 999999999` walked the range and ran to 80 GB. A range wider than the deck has ids is now walked from the ids, and the entries the deck never defines are counted rather than collected. - Whitespace inside a keyword is collapsed, so `*End Instance` is the card it looks like. STL: a zero attribute word is no colour under either convention. Read the Magics way to the letter it claims a facet colour of black, and an uncoloured Magics file came back black from end to end. Also: Medit refuses a float vertex reference rather than truncating it, the way it already refuses a float element reference, and its Dimension error no longer calls 3 a safety cap; Nastran reads CQUADX and reports a grounded CBUSH apart from the cards it has no shape for; Tecplot asks an attribute its shape through numpy, so a mesh built by hand with lists is skipped with a warning rather than an AttributeError; the shared tag helpers return an empty column's groups instead of raising, and report a group whose members cannot index the mesh.
PLY read gathered a whole element block with one structured dtype, which a list property has no fixed record width for: an 'edge' or 'vertex' element carrying one raised a bare KeyError. Such a block is now walked record by record, the scalar path kept for every file that does not need it. PLY write declared one header property for a multi-component element attribute and then wrote every component into the record, so a mesh read from a coloured STL wrote a file no reader could walk - and crashed outright in ASCII. Columns are spelled one property each, as a vertex attribute's already were. An attribute that is not one row per entity, or whose name is not a bare token, is skipped with a warning rather than indexed off the end. Gmsh wrote a NaN back as the token 'nan' under an entity count claiming a value for every element, so a field covering part of a mesh did not survive a round trip. Those rows are left out and the count drops with them, the tags of the rows that remain unchanged. A tag group holding an index past the end of the mesh no longer raises from inside numpy. Medit stepped over an undecoded section by name alone and left its records to the scan looking for the next section. That holds while a payload is numbers and stops holding at Identifier, whose value is a word: the word read as a section of its own. Sections are stepped over by their own length where that length is known. Nastran read a card carrying some of its mid-side grid points but not all as the linear element and dropped the rest without a word; the demotion is now counted and reported. Abaqus pinned the *INCLUDE boundary to each included file's directory as it descended, so a deck including parts/a.inp including ../common/nodes.inp was refused for leaving 'parts'. The boundary is the top deck's directory and stays there; nothing outside it is read. FLAC3D held ids in int64 arrays but parsed them as Python ints, so an id past that width surfaced as an OverflowError naming none of the file. The line iteration now really is lazy - splitlines() built the whole list the comment said it avoided. A codec that owns a shared extension now keeps its reads as well as its writes: a '.mesh' no sniffer claims goes to MFEM, whose own error names the problem, instead of stopping at the dispatcher.
Abaqus wrote a *Nset from a vertex tag without checking it: a stale index spelled an id no *Node card defines, and a float one was truncated onto another node. Both now go through member_indices and are reported, the way the element tags already were. A set name carrying a comma, an '=' or a line break is folded rather than ending the card early. Abaqus also cleared the deck's own node and element numbering on *PART or *INSTANCE, so a set out past *End Part reached nothing. The numbering is set aside and restored instead. PLY dropped element_attrs on its line elements: the edge records carried only their two ends, and a read stretched the face columns over them with NaN. The properties are declared on the edge records too and read back from them, so a value the mesh held on a line survives both ways; a property only one of the two elements declares is still NaN over the other. Binary face blocks are walked with struct rather than a numpy call per field, and a block whose records are all one width - which is nearly every file - is taken in a single structured read. Reading Armadillo.ply went from 1.15s to 0.44s. Medit held its references at 32 bits: a vertex reference past that saturated with a RuntimeWarning and an element reference raised a bare OverflowError. Both are read at 64 bits, which is what the ASCII format spells. The binary flavour's field really is 32 bits, so .meshb now reports a reference too wide for it instead of wrapping it onto another label, and validates vertex_attrs['ref'] the way the ASCII codec does - a float column was truncated and a short one escaped as a broadcast error. values_from_tags reports a group naming a label the column cannot hold rather than letting numpy raise. Tecplot wrote the token 'nan' for a variable carrying a missing value, which is a zone no reader loads. Such a variable is left out and named, the way the Gmsh data sections already did it. Gmsh split the file twice, once for the mesh sections and once for the data blocks; both come off one walk now. WKT reported a short TRIANGLE ring as a POLYGON's. Medit's sniff priority is spelled out rather than left to the module-name tiebreak, and the node counts Nastran and Medit each restated come from NODES_PER_ELEMENT.
Several writers declared one width and wrote another, which costs the records that follow rather than the field that did not fit. PLY kept two disagreeing dtype mappings: the header took an int64 column for 'int' while the body wrote eight bytes, so every field after it was framed wrong and the records behind it were lost. One decision now picks the type and casts the column, per column instead of per record - which also drops a numpy call per field and halves binary write time. A face count declared 'uchar' widens with the mesh rather than spelling a number one byte cannot hold, and an integer is written in full in ASCII instead of through a float format that turns a large one into an exponent. MFEM wrote any element it had no geometry for as geometry 2 and then all of its nodes, leaving the reader mid-record: two quadratic tetrahedra came back as a triangle and a tetrahedron, silently. Those elements are skipped and named, the way the sibling codecs do it. Gmsh declared whatever component count a field carried, where MSH2 allows 1, 3 or 9 and Gmsh refuses the rest; a field of another width is padded to the next legal one and the padding reported. The reader stays lenient. Medit refused nothing when a second Vertices section replaced the block its elements already indexed, which two matching counts hid outright, and said nothing when two ref_<n> groups named one element. Both are reported now; the overlap check lives in _tags so meshb reports it too. Also: the registry warns when two codecs claim one extension outright instead of leaving it to module walk order, an Abaqus set naming a member twice no longer reads as an element that never reached the file, and an assert in the PLY read path is no longer load-bearing under -O.
A line element of any width other than two was refused as its edge record was written, after the header and every face record were already on disk: the file left behind promised records its body did not carry, over whatever was at that path before. Every edge is measured now before the file is opened, in one pass over the offsets rather than a call per edge. A header may declare `face` more than once, and the binary reader kept only the last block's scalar columns while the element column grew with all of them, handing back an attribute shorter than the mesh. Both blocks are the one mesh now, and a property only a later block declares starts where that block does rather than flush at the first face - appending it flush moved every value onto a face that never carried it. The ASCII reader takes the same head padding, and both pad out to the element count so a property that stops early cannot be indexed off the end of. A header with no `end_header` line read forever: readline() returns nothing at the end of a file and goes on doing so. It ends the read, as does a header line shorter than its keyword spends, an element count that is not a number, a property name declared twice, and an unknown element whose records run past the end of the file - each of which raised an IndexError or a ValueError naming neither the element nor the file. The skipped-element walk resolves its record widths once instead of building a dtype per field per record.
A `ref_<n>` group whose members all index past the end of the mesh counted as a label named, so the writers took its all-zero column and relabelled every record 0 rather than leaving the field alone. It reaches no element and is reported alongside the groups that hold no indices at all. The labels come back as a named tuple now: six fields, four of them sets of names, is more than a positional unpack at a call site can be read back against, and both callers name what they report. A `.meshb` section declaring more records than the file carries raised out of numpy, naming neither the section nor the format, and a count wide enough asked for the allocation before saying so. The scan measures each decoded section against the file instead. The `.mesh` element cap was counted per section, and a mesh spreads its elements over as many sections as it holds types, so a cap every section passed on its own was one the file did not. A `Dimension` arriving after the `Vertices` block is refused: a vertex record is as wide as the dimension, so every coordinate came out of the wrong token, and a count that happens to divide hides that outright.
Two Abaqus tag groups can spell one card name: a name carrying ',', '=' or '*' folds those to '_', and Abaqus matches a set name without regard to case. Every member still reaches the file - Abaqus merges cards naming one set - but the groups stop being told apart, which is the kind of loss this codec reports everywhere else rather than leaving to be found on the way back in. Sharing an extension is a thing every owner has to say for itself. One that stayed silent while another shared kept nothing and said nothing: the whole alias read as settled because somebody had shared it. The warning names the owners that never listed it in SNIFF_EXTENSIONS. Which rows of a `$NodeData` / `$ElementData` field reach the file is the same for every field, so it is settled once rather than rebuilt per attribute. `_writable_variables` is annotated for the list its callers pass.
A PLY face and an OBJ 'f' record are a flat ring of vertex indices and nothing else: the file names no element type, so a reader has only the ring's width to go on and calls it a triangle at three, a quad at four and a polygon otherwise. Anything else a writer put in one - a tetra, a hexahedron, a quadratic_triangle, a polygon of three - kept its vertices and lost the type it was, and neither writer said so. Both name it now. The elements are still written: their vertices are the mesh, and refusing them would cost a caller the file they asked for. The rule and the report sit in one module, the way the tag convention does, so the two codecs cannot drift apart on what the readers actually make of a width. The pairs a mesh spreads over are counted by numpy rather than in a loop that would run once per element to say the same few things. An OBJ 'l' or 'p' record names geometry this codec has no element for and was stepped over as if it were a shading directive. The read counts them and says what the file held that the mesh does not.
skoudoro
commented
Aug 22, 2026
skoudoro
left a comment
skoudoro
left a comment
Collaborator
Author
There was a problem hiding this comment.
It took time, but look good now.
merging
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completes Priority 1 of the plan: P1.7 through P1.14, the correctness bugs in the formats polyxios already ships. Every item is test-first
1820 tests pass. 43 files changed.
What landed
CTETRAwith ten is aquadratic_tetra, with four atetra. Counting stops at the first blank field, since Nastran allows a mid-side point to be omitted and that card is a linear element.CPENTA's mid-side nodes are permuted into VTK order. Table extended to the one-dimensional families that used to read as an empty mesh (CBAR,CBEAM,CROD,CONROD,CTUBE,CBUSH,CGAP,CBEND,CVISC) plusCSHEAR,CTRAX3/6,CTRIAX6,CQUADX. ShellZOFFSread and written._medit.pyASCII codec..meshbgainedEdges,PrismsandPyramids; references becomeelement_attrs["ref"]plus oneelement_tags["ref_<n>"]group per value in both flavours.*NSET/*ELSET(block, standalone,GENERATE), several*NODEblocks,*INCLUDEwith a path-escape guard and depth cap,*SYSTEMtransforms,*PART/*INSTANCEmerged under their own numbering. Card table now resolves on the base name with modifier suffixes stripped, soCPS8Rreads asCPS8.element_type=write option; tags written back as*Nset/*Elset.$NodeData/$ElementDataread intovertex_attrs/element_attrsat any component count, scattered by the tag each row names, and written back.element edgewithvertex1/vertex2rather than a two-vertex face list. STL binary facet colours read intoelement_attrs["colors"]and written back.VARLOCATIONcell-centred variables parsed intoelement_attrsinstead of being refused; file and zone titles preserved inglobal_attrs.TRIANGLE/TIN/POLYHEDRALSURFACE,EMPTYincluded. Netgenfacedescriptorspromoted to a known section. FLAC3D read memory cut — see below.Three things worth a second look
FLAC3D #584 was not immune. The plan expected Cython to cover it. Measuring said otherwise: peak memory was 14x the file size, from Python lists of boxed floats and ints. Coordinates, connectivity, offsets and types now accumulate into
arrayrather thanlist, taking a 29 MB grid from 14x to 5.6x. A regression test guards a loose bound, so the boxed-object version cannot come back unnoticed.Medit high-order sections are deliberately skipped. libMeshb's own documentation (
Documentation/libMeshb7.tex:836) states the format fixes no node ordering for high-order elements — "there is, unfortunately, as many HO nodes ordering in each kind of elements as there are programmers" — and defers it to a companion*Orderingsection of Bezier indices. Decoding one without that section means guessing a permutation, which is the silent-corruption failure this codebase refuses everywhere else. They skip with a warning naming the section. Keyword codes are in place; filed as a P3 item.meshio's 18-node prism ordering is wrong, and that is recorded rather than matched. meshio maps Gmsh type 13 straight through to VTK type 32. polyxios permutes from VTK's own edge and face lists: Gmsh's face table runs
(0,1,4,3), (0,3,5,2), (1,2,5,4)where VTK runs(0,1,4,3), (1,2,5,4), (2,0,3,5), so the last two swap.test_issue_1517_meshio_does_not_permute_the_18_node_prismpins the disagreement so it is a deliberate divergence, not something to be rediscovered as a bug.One API-visible change
.meshis shared between MFEM and Medit ASCII, so it is no longer owned outright. The registry gained a way for a format to share its own extension: a codec listing an extension in bothEXTENSION(S)andSNIFF_EXTENSIONSis declaring that it shares it, and the resulting dispatcher takes its writer from whichever codec setsSNIFF_DEFAULT_WRITER.Reads of existing MFEM files are unaffected, and a bare
.meshwrite still emits MFEM..datkeeps its old behaviour: it belongs to no format, so a bare write there still raises and asks forfmt=.