Search Kue jobs by job data (using reds)

Problem: Consider a scenario where you are generating custom ids for each of your task. Now you want to search your jobs with these custom ids or any other data saved in the job.

By default kue doesn’t expose any method to search the jobs by the job data you saved. You can do so by using the api end point

GET /job/search?q=

Checkout https://github.com/Automattic/kue#get-jobsearchq

But this is not appropriate and code friendly to make an API call for every search. I looked up the KUE codebase for the inner functionalities of this API and wrote a simple function that will let you do the search without doing the API call.

You first need to install ‘reds’ which Kue internally uses for redis search operations.

npm install reds or yarn add reds

reds need you to index the data that you want to query about. Suppose you need to do search on customTaskId field that is in the data that you save in your job. While creating the job add searchKeys to the call.

queue.create('email', {
to: "abc@gmail.com",
customTaskId: "jbdh777733"
}).searchKeys(['customTaskId'])
.save((err) => {
})

Also search on custom data is disabled by default in Kue. You have set disableSearch: false when creating queue

const queue = kue.createQueue({ disableSearch: false })

Now the main function that will do the search on indexed ‘customTaskId’:

const getJobsByCustomTaskId = (customTaskId) => {
  const search = reds.createSearch(queue.client.getKey('search'))
  search.query(customTaskId).end((err, ids) => {
    if (err) return reject({ error: err.message })
    return ids
  })
})

Lookup into the reds documentation to understand better.

Let me walk you through the function:

queue.client.getKey(‘search’) return the params on which you indexed while creating the job.

You create a search instance by passing these params to reds.createSearch()

Then simply calling query on the search instance with the customTaskId returns the ids of the job with that customTaskId.

Now you can use these ids to do other operations exposed by the Kue sdk.

Hope it helps someone !!!