No way to send image / audio / video messages to a SquareChat (OpenChat)

[mimimiku778]

Summary

linejs can send media in a Talk (1:1 / group) via obs.uploadMediaByE2EE (recipient u… / c…), but there is no way to send an image / audio / video message to a SquareChat (OpenChat), and the intuitive "reserve an empty media message, then attach the obs data to its id" pattern is rejected by the server.

What fails

Reserving an empty media message through SquareService.sendMessage is rejected for IMAGE and AUDIO alike:

await client.base.square.sendMessage({
  squareChatMid: SQUARE_CHAT_MID,
  contentType: "IMAGE",          // or "AUDIO"
  contentMetadata: {},
});
// Request internal failed, sendMessage(/SQ1) ->
//   {"errorCode":"ILLEGAL_ARGUMENT","reason":"..."}

Uploading the obs object first and then calling sendMessage with the returned OID in contentMetadata is rejected the same way.

What works

Uploading through obs.uploadObjTalk with oid left undefined (the "reqseq" mode) makes the server create the message itself — no separate sendMessage is needed:

const blob = new Blob([bytes], { type: "image/png" /* or "audio/mp4" */ });
await client.base.obs.uploadObjTalk(
  SQUARE_CHAT_MID,   // square chat mid starts with "m" -> obs toType "g2"
  "image",           // or "audio"
  blob,
  undefined,         // oid unset => reqseq mode => server creates the message
  "image.png",       // or "voice.m4a"
);

Verified on a live OpenChat: both an image and a voice message are delivered this way. (One caveat: this obs-first path can't carry relatedMessageId, so media sent this way is never threaded as a reply.)

Proposed implementation

Adding first-class helpers to SquareChat (packages/linejs/client/features/square/mod.ts) is straightforward — it just wraps the obs-first upload above:

import type { ObjType } from "../../../base/obs/mod.ts";

// ... inside class SquareChat ...

/**
 * Sends an image to this OpenChat.
 *
 * OpenChat rejects the "reserve an empty IMAGE/AUDIO/VIDEO message via
 * sendMessage, then attach the obs object to its id" pattern with
 * ILLEGAL_ARGUMENT. Uploading the obs object with no `oid` ("reqseq" mode)
 * makes the server create the message itself. A consequence is that such a
 * message can't carry a `relatedMessageId`, so media sent this way is never
 * threaded as a reply.
 */
async sendImage(
  data: Blob,
  filename = "image",
): Promise<{ objId: string; objHash: string }> {
  return await this.#sendMedia("image", data, filename);
}

async sendVideo(
  data: Blob,
  filename = "video",
  durationMs?: number,
): Promise<{ objId: string; objHash: string }> {
  return await this.#sendMedia("video", data, filename, durationMs);
}

async sendAudio(
  data: Blob,
  filename = "voice.m4a",
  durationMs?: number,
): Promise<{ objId: string; objHash: string }> {
  return await this.#sendMedia("audio", data, filename, durationMs);
}

async #sendMedia(
  type: ObjType,
  data: Blob,
  filename: string,
  durationMs?: number,
): Promise<{ objId: string; objHash: string }> {
  const { objId, objHash } = await this.#client.base.obs.uploadObjTalk(
    this.raw.squareChatMid,
    type,
    data,
    undefined, // oid unset => reqseq mode => the server creates the message
    filename,
    durationMs, // forwarded to the obs `duration` param (see #205)
  );
  return { objId, objHash };
}

The durationMs argument depends on #205 (which makes uploadObjTalk forward a real audio/video duration instead of hardcoding "1919"); without it audio/video would always display ~1.9s. I'm happy to send this as a PR if the API shape looks good.

Environment

• linejs 3.1.4
• device: DESKTOPWIN
• target: SquareChat (OpenChat) send path

[EdamAme-x]

looks good

[mimimiku778]

Opened #206 for this (stacked on #205).