Skip to content
46 changes: 44 additions & 2 deletions src/agent/custom-http-agent.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
import {
Certificate,
Expiry,
HttpAgent,
Nonce,
defaultStrategy,
lookupResultToBuffer,
makeExpiryTransform,
makeNonceTransform,
pollForResponse as pollForResponseAgent,
type CallRequest,
Expand All @@ -13,7 +15,10 @@ import {
import {bufFromBufLike} from '@dfinity/candid';
import {Principal} from '@dfinity/principal';
import {base64ToUint8Array, isNullish, nonNullish} from '@dfinity/utils';
import {DEFAULT_EXPIRY_DURATION} from '../constants/http-agent.constants';
Comment thread
roman-nazaruk marked this conversation as resolved.
Outdated
import type {IcrcCallCanisterRequestParams} from '../types/icrc-requests';
import {generateHash} from '../utils/crypto.utils';
import {expiryToMs} from '../utils/custom-http-agent.utils';

export type CustomHttpAgentResponse = Pick<Required<SubmitResponse>, 'requestDetails'> & {
certificate: Certificate;
Expand All @@ -30,9 +35,11 @@ export class UndefinedRootKeyError extends Error {}
// Therefore, it is cleaner in my opinion to encapsulate the agent rather than extend it.
export class CustomHttpAgent {
readonly #agent: HttpAgent;
#cache: Map<string, Expiry>;

private constructor(agent: HttpAgent) {
this.#agent = agent;
this.#cache = new Map();
}

static async create(
Expand All @@ -55,9 +62,14 @@ export class CustomHttpAgent {
arg,
canisterId,
method: methodName,
nonce
}: Omit<IcrcCallCanisterRequestParams, 'sender'>): Promise<CustomHttpAgentResponse> => {
nonce,
sender
}: IcrcCallCanisterRequestParams): Promise<CustomHttpAgentResponse> => {
const hash = nonce ? await generateHash({ canisterId, sender, method: methodName, arg, nonce }) : null;
const ingressExpiry = hash ? this.getIngressExpiry(hash) : DEFAULT_EXPIRY_DURATION;

this.attachRequestNonce({nonce});
this.attachAddTransformExpiry(ingressExpiry);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't have to set a cached expiry, I would let agent-js set the expiry because they're doing some time sync vodoo.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we always need to set a transform because we cache the agent. If we don’t, we might make a call with the expiry of a previous call. Unless agent-js provides a way to remove a transformer or we re-create an instance—but I guess the first option doesn’t exist, and the second is less performant and also complicates the code.

@tmu0 tmu0 Mar 17, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, good point. With the same argument, the current implementation of attachRequestNonce() also shouldn't work because it conditionally sets the transform. WDYT of the following "creative" approach:

  1. We write a custom implementation of HttpAgentRequestTransformFn that takes care of setting the nonce and the expiry and manages the expiry cache. (Behavior described in step 4).
  2. In CustomHttpAgent.constructor(agent) we set it via agent.addTransform()
  3. In CustomHttpAgent.request() we append the nonce and the length of it to the method name. If no nonce is specified, we just append length 0.
  4. In our custom transform implementation, we remove the nonce and length from the method name.
    • If nonce length is 0, we're done.
    • Otherwise we set the chosen nonce in the request, extract the other request parameters necessary to construct the cache key and check the expiry cache.
      • On cache miss we add the expiry from the request to the cache.
      • On cache hit we replace the expiry in the request with the one from the cache.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the current implementation of attachRequestNonce() also shouldn't work

We recently merged a PR to overcome this issue with the approach I mentioned (#502). Not that I like it—just sharing the information to let you know it should be fine for now.

following "creative" approach

I like the idea of implementing our custom version of HttpAgentRequestTransformFn. This way, both nonce and expiry handling can be scoped within the same function and module, improving maintainability and testing.

As for appending the nonce and its length to the method name—why not? However, if we can find a way to handle this in a less hacky manner (😉), not again, but fundamentally speaking sure yes, I think having one custom function is a good idea.


const {requestDetails, ...restResponse} = await this.#agent.call(canisterId, {
methodName,
Expand All @@ -77,6 +89,8 @@ export class CustomHttpAgent {
canisterId
});

this.cleanCacheAfterCall();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we agreed on not removing entries from the cache, so this method can be deleted.


// I assume that if we get a result at this point, it means we can respond to the caller.
// However, this is not how it's handled in Agent-js. For some reason, regardless of whether they get a result at this point or not, if the response has a status of 202, they overwrite the result with pollForResponse, which seems incorrect.
// That is why we return the result if we get one.
Expand Down Expand Up @@ -210,4 +224,32 @@ export class CustomHttpAgent {
makeNonceTransform((): Nonce => base64ToUint8Array(nonce) as Nonce)
);
}
private attachAddTransformExpiry(expiry: number): void {
this.#agent.addTransform('update', makeExpiryTransform(expiry));
}

private getIngressExpiry(hash: string): number {
const existingExpiry = this.#cache.get(hash);

// TODO: Should we try with a new expiry or reject the request if the timestamp is expired?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After thinking a bit more about it, it might be better to reject the request if the timestamp is expired instead of generating a new one (and also not removing expired timestamps from the cache), WDYT?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also rather like rejection. It's also easier/cleaner to implement.

return (!existingExpiry || expiryToMs(existingExpiry) <= Date.now())
? this.createAndStoreNewExpiry(hash)
: expiryToMs(existingExpiry) - Date.now();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is dangerous, because there is no guarantee that makeExpiryTransform will set the same expiry (it will call Date.now() again to convert back from delta to expiry, which might result in a slightly different value). We should implement our own HttpAgentRequestTransformFn that will directly receive an Expiry (instead of a delta) and sets it.

}

private createAndStoreNewExpiry(hash: string): number {
const newExpiry = new Expiry(DEFAULT_EXPIRY_DURATION);
this.#cache.set(hash, newExpiry);
return DEFAULT_EXPIRY_DURATION;
}

private cleanCacheAfterCall(): void {
const now = Date.now();

for (const [key, expiry] of this.#cache.entries()) {
if (expiryToMs(expiry) <= now) {
this.#cache.delete(key);
}
}
}
}
3 changes: 2 additions & 1 deletion src/api/signer.api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,14 @@ export class SignerApi extends Icrc21Canister {
async call({
owner,
host,
params: {canisterId, method, arg, nonce}
params: {canisterId, method, arg, nonce, sender}
}: {
params: IcrcCallCanisterRequestParams;
} & SignerOptions): Promise<IcrcCallCanisterResult> {
const agent = await this.getAgent({host, owner});

const result = await agent.request({
sender,
canisterId,
method,
arg,
Expand Down
1 change: 1 addition & 0 deletions src/constants/http-agent.constants.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export const DEFAULT_EXPIRY_DURATION = 5 * 60 * 1000;
11 changes: 11 additions & 0 deletions src/utils/crypto.utils.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import {uint8ArrayToHexString} from '@dfinity/utils';
import {IcrcCallCanisterRequestParams} from 'src/types/icrc-requests';

export async function generateHash(params: IcrcCallCanisterRequestParams): Promise<string> {
const jsonString = JSON.stringify(params, Object.keys(params).sort());

const dataBuffer = new TextEncoder().encode(jsonString);
const hashBuffer = await crypto.subtle.digest('SHA-256', dataBuffer);

return uint8ArrayToHexString(new Uint8Array(hashBuffer));
}
5 changes: 5 additions & 0 deletions src/utils/custom-http-agent.utils.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import {Expiry} from '@dfinity/agent';

export function expiryToMs(expiry: Expiry): number {
return Number(expiry['_value'] / 1000000n);
}