Skip to content

Add streaming support - #57

Open
bglusman wants to merge 3 commits into
HGInsights:mainfrom
bglusman:add_streaming_support
Open

Add streaming support#57
bglusman wants to merge 3 commits into
HGInsights:mainfrom
bglusman:add_streaming_support

Conversation

@bglusman

@bglusman bglusman commented Feb 6, 2025

Copy link
Copy Markdown

FIrst pass at resolving #51 I opened quite a while back now, to be totally frank Claude wrote a lot of this code via Cursor or Windsurf, but I definitely participated a fair bit 🤣 I think it at least serves as a good start as it seems to work and preserve all the old behavior/passes the old tests and the new ones... I'm not totally sure about the async interaction/didn't test that exlicitly, so maybe someone who knows the code around async can have a look/suggest some test or concerns around interactions there? 🤷‍♂️
cc @forest since we discussed last year and @vuphamcs since he reacted there, expressing some interest... maybe of interest to @pdgonzalez872 also and @jeremyowensboggs since I discussed with him/may have been his idea originally...

@bglusman
bglusman requested a review from forest as a code owner February 6, 2025 21:54
defp build_pipeline(request, opts) do
disable_polling = Keyword.fetch!(opts, :async)
async? = Keyword.fetch!(opts, :async)
streaming? = Keyword.get(opts, :streaming, false)

@bglusman bglusman Feb 7, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm should this ALSO be fetch! because of how NimbleOptions will build opts and take care of the default value I guess?

Comment on lines +62 to +66
Enum.zip_reduce(types, row, %{}, fn type, value, result ->
column_name = maybe_downcased_column_name(type, downcase_column_names)
column_value = decode(type, value)
Map.put(result, column_name, column_value)
end)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could DRY this up with the above function head of decode_data_rows

@forest

forest commented Feb 13, 2025

Copy link
Copy Markdown
Collaborator

@aschreifels FYI. Not sure how much the team is investing in SF these days, but this is the start of a decent feature.

@forest
forest requested a review from aschreifels February 13, 2025 01:51
@aschreifels

Copy link
Copy Markdown

@forest No investment currently, as you can see I am 13 days late to this from you and 3 weeks late to @bglusman opening the PR...

I will take a look this week and get back here. Thanks for this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants