You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Reconsider policy on blocking queries containing the string "pragma" #1618
Table-valued functions exist only for PRAGMAs that return results and that have no side-effects.
So it's possible I'm being overly paranoid here after all: what I want to block here is people running things like PRAGMA case_sensitive_like = 1 which could affect the global state for that connection and cause unexpected behaviour later on.
So maybe I should allow all pragma functions. I previously allowed an allow-list of them in:
changed the title [-]Checks for allowed PRAGMAs prevent legitimate queries[/-][+]Reconsider policy on blocking queries containing the string "pragma"[/+]on Feb 2, 2022
So I could use that list to create a much more specific regular expression, which would then allow the word "pragma" to be used more freely while still protecting against any known pragma function being called.
I also need to consider if supposedly harmless side-effect free pragma functions could be used to work around the Datasette permissions system. My hunch is that wouldn't be a problem, because if you're allowing arbitrary SQL queries you're already letting people ignore the permissions system.
One example:
sqlite-utils fixtures.db 'pragma database_list' -t
seq name file
----- ------ ------------------------------------------------------
0 main /Users/simon/Dropbox/Development/datasette/fixtures.db
Thanks for looking into this. It might have been nice if explain surfaced these function calls. Looks like explain query plan does, but only for basic queries.
sqlite-utils fixtures.db 'explain query plan select * from pragma_function_list(), pragma_database_list(), pragma_module_list()' -t
id parent notused detail
---- -------- --------- ------------------------------------------------
4 0 0 SCAN pragma_function_list VIRTUAL TABLE INDEX 0:
8 0 0 SCAN pragma_database_list VIRTUAL TABLE INDEX 0:
12 0 0 SCAN pragma_module_list VIRTUAL TABLE INDEX 0:
sqlite-utils fixtures.db 'explain query plan select * from pragma_function_list() as fl, pragma_database_list() as dl, pragma_module_list() as ml' -t
id parent notused detail
---- -------- --------- ------------------------------
4 0 0 SCAN fl VIRTUAL TABLE INDEX 0:
8 0 0 SCAN dl VIRTUAL TABLE INDEX 0:
12 0 0 SCAN ml VIRTUAL TABLE INDEX 0:
Activity
simonw commentedon Feb 2, 2022
There's a workaround for this at the moment, which is to use parameterized SQL queries. For example, this:
https://fivethirtyeight.datasettes.com/polls?sql=select+*+from+books+where+title+%3D+%3Atitle&title=The+Pragmatic+Programmer
So the SQL query is
select * from books where title = :titleand then&title=...is added to the URL.The reason behind the quite aggressive pragma filtering is that SQLite allows you to execute pragmas using function calls, like this one:
These can be nested arbitrarily deeply in sub-queries, so it's difficult to write a regular expression that will definitely catch them.
I'm open to relaxing the regex a bit, but I need to be very confident that it's safe to do so.
simonw commentedon Feb 2, 2022
But... I just noticed something I had missed in the docs for https://www.sqlite.org/pragma.html#pragfunc
So it's possible I'm being overly paranoid here after all: what I want to block here is people running things like
PRAGMA case_sensitive_like = 1which could affect the global state for that connection and cause unexpected behaviour later on.So maybe I should allow all pragma functions. I previously allowed an allow-list of them in:
[-]Checks for allowed PRAGMAs prevent legitimate queries[/-][+]Reconsider policy on blocking queries containing the string "pragma"[/+]simonw commentedon Feb 2, 2022
I also just realized that
pragma pragma_listcan be used to generate a list of all known pragmas for the connection:So I could use that list to create a much more specific regular expression, which would then allow the word "pragma" to be used more freely while still protecting against any known pragma function being called.
simonw commentedon Feb 2, 2022
I also need to consider if supposedly harmless side-effect free pragma functions could be used to work around the Datasette permissions system. My hunch is that wouldn't be a problem, because if you're allowing arbitrary SQL queries you're already letting people ignore the permissions system.
One example:
Though it looks like I already allow-listed that one in #761: https://latest.datasette.io/_memory?sql=select+*+from+pragma_database_list%28%29
simonw commentedon Feb 2, 2022
I was hoping that
explain select ...might be able to easily spot when people are calling PRAGMA functions, but this output doesn't look very helpful:strada commentedon Feb 2, 2022
Thanks for looking into this. It might have been nice if
explainsurfaced these function calls. Looks likeexplain query plandoes, but only for basic queries.