waitForResponse: false is broken

Bugs
1

Issue Description: What’s happening?

When invoking a task using "waitForResponse": false, the task is never invoked.

Reproduce the Issue: What steps can someone take to replicate the problem?

  1. Create a Trigger that invokes a task with waitForResponse: false
  2. Cause Trigger to fire.
  3. Task never gets invoked.

Expected Behavior: What did you expect to happen?

I expect the task to be invoked and the Function invoking it to not wait for it to finish.

Actual Behavior: What actually happened?

The task was never invoked.

More details or screenshot

This code never invokes the function at all.

await ctx.invokeFunction(
      "AI_REPORTS_QUERY",
      {
        data: { type: "UPDATE", reports: chunk, aIReportsScoreVariables }
      },
      {
        waitForResponse: false
      }
    );

This code invokes the function just fine, but waits for it to finish which I don’t want.

await ctx.invokeFunction(
      "AI_REPORTS_QUERY",
      {
        data: { type: "UPDATE", reports: chunk, aIReportsScoreVariables }
      },
      {
        waitForResponse: true
      }
    );

@sebastian.scholl The docs also says that “options.waitForResponse (default: false)” Which is not currerntly correct. It defaults to “true” as leaving out the 3rd options argument, makes it wait for the Task to finish each time (and the task is invoked). However again my trigger fails due to the 20s timeout.

Docs: https://docs.8base.com/docs/8base-console/custom-functions/tasks

2

Hey @MarkLyck! So we’ve ran a few tests and waitForResponse: false is working on our end. My best guess is that you’re still running into a timeout issue where the operation for each task is taking longer than 20 seconds, however since this runs on a separate container you don’t get any error messages.

Can you try reducing you chunk size?

3

@sebastian.scholl

This doesn’t just affect my big task. But also my tiny tasks such as adding or removing a single Email from my SendGrid database. (this task only takes 900ms) and runs fine when “waitForResponse” is set to “true”. When it’s false it doesn’t run at all.

This definitely isn’t related to the 20-second timeout.

This is consistently broken with all my tasks, only one of them takes a while to complete, and. even that I split up into 100 item chunks that takes about. 5 seconds each.

The main trigger is the one that times out at 20 seconds. (because it has to wait for each ~5 second. task to finish as I can’t turn waitForResponse to false :frowning:

4

@sebastian.scholl Here’s a full code example of this failing:

Trigger - Newsletter.after create function

type TriggerResult = {
  data: {
    success: boolean;
  };
  errors: Array<object>;
};

export default async (event: any, ctx: any): Promise<TriggerResult> => {
  const { email } = event.data;

  await ctx.invokeFunction("EMAIL_LIST_ADD_CONTACT", {
    data: {
      email,
      listIDs: [process.env.EMAIL_LIST_ID_NEWSLETTER]
    }
  }, {
    waitForResponse: false
  });

  return {
    data: {
      success: true
    },
    errors: []
  };
};

Task function called EMAIL_LIST_ADD_CONTACT

const client = require("@sendgrid/client");
client.setApiKey(process.env.SENDGRID_API_KEY);

type EventType = {
  data: {
    listIDs: [string];
    email: string;
    firstName: string;
    lastName: string;
  };
};

type TaskResult = {
  data: {
    success: boolean;
  };
};

export default async (event: EventType, ctx: any): Promise<TaskResult> => {
  const { listIDs, email, firstName, lastName = "" } = event.data;
  console.log(`Adding "${email}" to lists [${listIDs}]`);

  const options = {
    method: "PUT",
    url: "/v3/marketing/contacts",
    body: {
      list_ids: listIDs,
      contacts: [
        {
          email,
          first_name: firstName,
          last_name: lastName
        }
      ]
    }
  };

  return client
    .request(options)
    .then(([_response, _body]) => {
      console.log(`Added "${email}" to email lists: [${listIDs}]`);

      return {
        data: {
          success: true
        },
        errors: []
      };
    })
    .catch(err => ({ data: { success: false }, errors: [err] }));
};

When the above Trigger runs the Task is never invoked.

However, if I change the waitForResponse to “true” it runs fine and the task runs and finishes in around ~900 milliseconds.

5

@sebastian.scholl @evgeny.semushin am I doing anything wrong in the broken example above?

6

I don’t see any issues with your example, it should work fine. Also I just tried the similar example with my workspace and it worked as expected.
Can we try to run an experiment:

  1. in Newsletter.after add console.log("testAsync: Newsletter.after") after const { email } = event.data;
  2. make waitForResponse: false
  3. in EMAIL_LIST_ADD_CONTACT task add console.log("testAsync: EMAIL_LIST_ADD_CONTACT") before your console.log(Adding “${email}” to lists [${listIDs}]);

Then try to add a record to Newsletter table and post message here so I can review logs and try to understand what’s happening.

1 Like
7

Hmm oddly enough, running it today it worked fine after seeing this fail on several occasions in 2 workspaces in 4 different functions.

Maybe I have just been writing it slightly wrong all other times. :thinking:

@evgeny.semushin thanks anyway though! Hopefully it just works for the other scenarios I need it as well.

8

@evgeny.semushin Going back to the first case of waitForResponse: false not working though.

That still doesn’t work for me :confused: and I can’t see anything different I’m doing except doing it in a loop instead of just once?

in UPDATE_AI_REPORTS Task function.

for (const chunk of chunkedReportsToUpdate) {
    console.log("start chunk");
    await ctx.invokeFunction(
      "AI_REPORTS_QUERY",
      {
        data: { type: "UPDATE", reports: chunk, aIReportsScoreVariables }
      },
      { waitForResponse: false }
    );
    console.log("end chunk");
  }

The loop is running inside the main ASYNC function, so it should have access to await (this is also confirmed by the fact when setting waitForResponse to false, it runs 1 at a time as expected)

“start chunk” & “end chunk” logs as expected 12 times in total. But it never invokes the other function.

in AI_REPORTS_QUERY Task function:

export default async (event: any, ctx: any): Promise<TaskResult> => {
  const { type, reports, aIReportsScoreVariables } = event.data;
  console.log("AI Reports Query called");

  if (type === "UPDATE" && reports.length) {
    console.log("aIReportsToUpdate.length", reports.length);

    const UPDATE_AI_REPORTS_MUTATION = generateUpdateMutation(reports);
    console.log("before GraphQL");
    await graphQLClient.request(UPDATE_AI_REPORTS_MUTATION, {
      ...aIReportsScoreVariables
    });
    console.log("after GraphQL");
  }

  return {
    data: {
      result: `Task recieved: ${event.data.foo}`
    }
  };
};

Is this not working because of the for of loop? The AI_REPORTS_QUERY never gets invoked when waitForResponse: false is set. When it’s set to true or just not defined. It runs about 5 instances of AI_REPORTS_QUERY one at a time, but eventually the Main UPDATE_AI_REPORTS task times out after the uneditable 20 second timeout and the other task stops being invoked before I finished looping through and updating all the data.

Am I doing anything wrong with this? Seems the same as the other one except there’s a loop involved. But I specifically used a for of loop which supports async / await, and not e.g. forEach which doesn’t.

workspace ID: ck34xg5y400ft01l2c12e53x8

9

It looks like you are hitting lambda limit:
https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html
Invocation payload (request and response)
6 MB (synchronous)
256 KB (asynchronous)

Can you please try to make chunks smaller and try again

1 Like
10

Ohhh I didn’t know that.

I’ll have to rework some stuff then. But that explains a lot!

For testing purposes right now it sends over ALL 2.5k variables it needs, but then only updates 200 records.

If I rework it to just generate variables for each chunk and not all then it’s probably within the limit :slight_smile:

Thank you once again!

11

@evgeny.semushin calling the async functions with smaller amounts of data worked.

Doing 100 updates at a time now and seems to run consistently! Thank you