-
Notifications
You must be signed in to change notification settings - Fork 188
utils: port csv_dequote.pl to Python #1728
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Valyrian-Code
wants to merge
6
commits into
OSGeo:grass8
Choose a base branch
from
Valyrian-Code:port-csv-dequote-to-python
base: grass8
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+91
−66
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
1b850e4
utils: port csv_dequote.pl to Python
Valyrian-Code 8404f8e
Merge branch 'grass8' into port-csv-dequote-to-python
Valyrian-Code 661a589
Merge branch 'grass8' into port-csv-dequote-to-python
echoix f312f6c
Merge branch 'grass8' into port-csv-dequote-to-python
Valyrian-Code 962c52c
csv_dequote: use exclusive "x" mode instead of check-then-open
Valyrian-Code 0c20348
Merge branch 'grass8' into port-csv-dequote-to-python
echoix File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| #!/usr/bin/env python3 | ||
| """csv_dequote.py. | ||
|
|
||
| Take a .csv file with quoted strings, convert the real comma delimiters | ||
| to pipes (|) and strip away the double quote chars. | ||
|
|
||
| Author: Hamish Bowman, Dunedin, New Zealand, June 2012 (original Perl) | ||
| Python port: 2026 | ||
| (c) 2012 M. Hamish Bowman, and the GRASS Development Team | ||
| License: GNU GPL >=2. See the GPL.TXT file which comes with GRASS | ||
| for details. | ||
|
|
||
| USAGE: csv_dequote.py infile.csv [outfile.psv] | ||
|
|
||
| if outfile is not given it will take the basename of the input | ||
| file and give it the .psv extension ('Pipe Sep Vars'). | ||
| """ | ||
|
|
||
| import csv | ||
| import os | ||
| import sys | ||
|
|
||
| OUTSEP = "|" | ||
|
|
||
|
|
||
| def main(argv): | ||
| if len(argv) < 2 or len(argv) > 3: | ||
| sys.stderr.write("USAGE: csv_dequote.py infile.csv [outfile.psv]\n") | ||
| return 1 | ||
|
|
||
| infile = argv[1] | ||
| outfile = argv[2] if len(argv) == 3 else None | ||
|
|
||
| if outfile is None: | ||
| # Match the Perl behavior: File::Basename::fileparse + "$file.psv" | ||
| # drops the directory component. | ||
| base = os.path.basename(infile) | ||
| base_no_ext, _ = os.path.splitext(base) | ||
| outfile = base_no_ext + ".psv" | ||
|
|
||
| try: | ||
| fin = open(infile, newline="") | ||
| except OSError as exc: | ||
| sys.stderr.write("{}: {}\n".format(infile, exc.strerror)) | ||
| return 1 | ||
|
|
||
| try: | ||
| # "x" creates the file atomically and fails if it already exists, | ||
| # so we never overwrite an existing file (no check-then-open race). | ||
| fout = open(outfile, "x", newline="") | ||
| except FileExistsError: | ||
| fin.close() | ||
| sys.stderr.write( | ||
| 'ERROR: "{}" already exists.\n' | ||
| " Will not overwrite; aborting.\n".format(outfile) | ||
| ) | ||
| return 1 | ||
| except OSError as exc: | ||
| fin.close() | ||
| sys.stderr.write("{}: {}\n".format(outfile, exc.strerror)) | ||
| return 1 | ||
|
|
||
| had_parse_error = False | ||
| try: | ||
| reader = csv.reader(fin) | ||
| # csv.Error is raised by next() on parse failures, not by the writes. | ||
| # Drive the reader explicitly so we can catch and continue per row. | ||
| while True: | ||
| try: | ||
| row = next(reader) | ||
| except StopIteration: | ||
| break | ||
| except csv.Error as exc: | ||
| sys.stderr.write( | ||
| "Unable to parse CSV at line {} in {}: {}\n".format( | ||
| reader.line_num, infile, exc | ||
| ) | ||
| ) | ||
| had_parse_error = True | ||
| continue | ||
| fout.write(OUTSEP.join(row)) | ||
| fout.write("\n") | ||
| finally: | ||
| fin.close() | ||
| fout.close() | ||
|
|
||
| return 1 if had_parse_error else 0 | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main(sys.argv)) | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.