- Add the background task¶
- Caveat¶
- Create a task function¶
- Running a sql query in the background using mysql
- Technical details¶
- Using backgroundtasks¶
- Как найти задания, работающие в фоновом режиме?
- Как переместить процесс из фонового режима в приоритетный?
- Как прекратить выполнение заданий, работающих в фоновом режиме?
- Примеры
- Синтаксис
- Dependency injection¶
- Recap¶
Add the background task¶
Inside of your path operation function, pass your task function to the background tasks object with the method .add_task():
.add_task() receives as arguments:
- A task function to be run in the background (
write_notification
). - Any sequence of arguments that should be passed to the task function in order (
email
). - Any keyword arguments that should be passed to the task function (
message="some notification"
).
Caveat¶
If you need to perform heavy background computation and you don’t necessarily need it to be run by the same process (for example, you don’t need to share memory, variables, etc), you might benefit from using other bigger tools like Celery.
They tend to require more complex configurations, a message/job queue manager, like RabbitMQ or Redis, but they allow you to run background tasks in multiple processes, and especially, in multiple servers.
To see an example, check the Project Generators, they all include Celery already configured.
But if you need to access variables and objects from the same FastAPI app, or you need to perform small background tasks (like sending an email notification), you can simply just use BackgroundTasks.
Create a task function¶
Create a function to be run as the background task.
It is just a standard function that can receive parameters.
It can be an async def or normal def function, FastAPI will know how to handle it correctly.
In this case, the task function will write to a file (simulating sending an email).
And as the write operation doesn’t use async and await, we define the function with normal def:
Running a sql query in the background using mysql
If you find you are experiencing issues using nohup
with mysql
– I found an alternative solution inspired by @dan08’s answer on this page, as well as @Node’s answer about putting an already running process into the background from another question on SO.
I had found that when trying the suggestion of using nohup mysql...
to ‘background’ a long-running mysql
command, that the remote MySQL server I was attempting to connect to refused the connection. When running the same mysql
command without prefixing it with nohup
, the connection succeeded and the mysql
command worked as expected.
As such I looked for alternative methods of achieving the same end-result of being able to run a long-running SQL command (or set of SQL commands) without concerns about the process terminating if I disconnected from the shell session or if a networking issue caused a disconnection from the remote server. This research led to the working solution proposed below:
- Run your
mysql
command as though you would like to run it in the foreground; for example:mysql <options> -h <host> -u <user> -p -e "source
/absolute/path/to/commands.sql" <database name> 2>&1 >
./mysql_output.log - When asked, input the relevant password at the MySQL command prompt for the specified
<user>
(this is triggered by the inclusion of the-p
flag in themysql
command above). - Immediately press Ctrl Z to suspend the current process, in this case
mysql
. - Run the
bg
command to move the currently suspended process into the background, and resume its processing. - Run the
jobs
command to list the jobs currently running in the shell session (you may only have one job listed if you don’t have any other background processes running in the current session). From this list note the job number formysql
, which will likely be1
if you only have one job running. - Last, but not least, run the
disown -h %<job number>
command, thusdisown -h %1
(assuming the job number obtained in step #5 above was1
). This last command is very important, as it disassociates the current shell session from ourmysql
process which we have just moved into the background, so that themysql
process does not get killed when the terminal shell session ends. Without the call todisown
, yourmysql
process that is now running in the background could still get killed if your terminal session ends.
The above sequence of commands effectively mimics a call to nohup mysql...
by allowing you to move a long-running call to mysql
into the background and not have the process terminated if you close your terminal session or get disconnected from the remote server. If you find that running nohup mysql...
works in your circumstances that is great as it is certainly the easier and quicker approach, but at least in our application/server environment, we were unable to use nohup mysql...
successfully.
The above sequence should also give you more control over executing the mysql
command as you are initially running the command in the foreground; for instance running mysql
in the foreground will allow you to enter your MySQL connection password securely at the MySQL prompt without having to provide the password as plain-text on the command line (which may be an important security consideration in some hosting/server environments, especially shared hosting where passwords entered on the command line would likely end up in log files such as ~/.bash_history or similar). Additionally, running the command in the foreground, will allow you to see and respond to any connection or other errors before the MySQL command gets moved into the background.
Credit must also go @dan08’s answer on this page, and to @Node for his answer regarding placing an already running foreground process into the background, for helping inspire the solution proposed here.
Hopefully this proposed solution will help others who find themselves in a similar situation, where running nohup mysql...
does not work as expected.
Technical details¶
The class BackgroundTasks comes directly from starlette.background.
It is imported/included directly into FastAPI so that you can import it from fastapi and avoid accidentally importing the alternative BackgroundTask (without the s at the end) from starlette.background.
By only using BackgroundTasks (and not BackgroundTask), it’s then possible to use it as a path operation function parameter and have FastAPI handle the rest for you, just like when using the Request object directly.
It’s still possible to use BackgroundTask alone in FastAPI, but you have to create the object in your code and return a Starlette Response including it.
You can see more details in Starlette’s official docs for Background Tasks.
Using backgroundtasks¶
First, import BackgroundTasks and define a parameter in your path operation function with a type declaration of BackgroundTasks:
FastAPI will create the object of type BackgroundTasks for you and pass it as that parameter.
Как найти задания, работающие в фоновом режиме?
Выполните следующую команду:jobs
Пример вывода данных:
[1]- Running find / -iname "*.c" 2> /dev/null > /tmp/output.txt & [2] Running grep -R "hostNamed" / 2> /dev/null > /tmp/grep.txt &
Где[1] и [2] идентификаторы заданий.Чтобы отобразить идентификаторы процессов для идентификаторов заданий помимо стандартных сведений, передайте параметр -l: jobs –l
Пример вывода данных:
[1]- 7307 Running find / -iname "*.c" 2> /dev/null > /tmp/output.txt & [2] 7324 Running grep -R "hostNamed" / 2> /dev/null > /tmp/grep.txt &
Чтобы отобразить только идентификаторы процессов, введите:jobs -p
Пример вывода данных:
7307 7324
Как переместить процесс из фонового режима в приоритетный?
Синтаксис: %JOB-ID
ИЛИ fg JOB-ID
Сначала следует вывести список текущих заданий с помощью команды jobs. Для этого введите: jobs -l
Пример вывода данных:
[1]- 7307 Running find / -iname "*.c" 2> /dev/null > /tmp/output.txt & [2] 7324 Running grep -R "hostNamed" / 2> /dev/null > /tmp/grep.txt &
Чтобы перевести задание job id #2 в приоритетный режим, введите: %2 ИЛИ используйте команду: fg 2
Пример вывода данных:
grep -R "hostNamed" / 2> /dev/null > /tmp/grep.txt
Чтобы перевести это задание обратно в фоновый режим, нажмите клавишу Z, удерживая клавишу CTRL. При этом выполнение текущего приоритетного задания будет приостановлено. Введите следующую команду, чтобы перевести задание в фоновый режим:%2 &ИЛИ используйте команду: bgТеперь командное задание grep будет работать в фоновом режиме.
Как прекратить выполнение заданий, работающих в фоновом режиме?
Для принудительного или постепенного завершения процесса следует использовать команду kill. Синтаксис выглядит следующим образом:
kill PID kill -15 PID kill -9 PID killall process-Name-Here killall -15 process-Name-Here killall -9 process-Name-Here
Примеры
Чтобы перевести команду ls в фоновый режим, введите:$ ls *.py > output.txt &
Чтобы перевести следующую команду поиска в фоновый режим, добавьте символ «&» в конец командной строки:find . -iname “*.mp3” > myplaylist.txt &
Синтаксис
Задачу (например, команду или скрипт) можно перевести в фоновый режим, добавив символ «&» в конец командной строки. Этот оператор переводит команду в фоновый режим и освобождает место в терминале. Команда, выполняемая в фоновом режиме, называется заданием. Во время работы фоновой команды можно выполнять любые другие команды. Синтаксис выглядит следующим образом:
command & script-name & /path/to/command arg1 arg2 & command-1 | command-2 arg1 & command-1 | command-2 -arg1 -arg2 >/path/to/output &
Dependency injection¶
Using BackgroundTasks also works with the dependency injection system, you can declare a parameter of type BackgroundTasks at multiple levels: in a path operation function, in a dependency (dependable), in a sub-dependency, etc.
FastAPI knows what to do in each case and how to re-use the same object, so that all the background tasks are merged together and are run in the background afterwards:
In this example, the messages will be written to the log.txt file after the response is sent.
If there was a query in the request, it will be written to the log in a background task.
And then another background task generated at the path operation function will write a message using the email path parameter.
Recap¶
Import and use BackgroundTasks with parameters in path operation functions and dependencies to add background tasks.