Running a Python Application as a Windows Service

By:   |   Updated: 2022-07-11   |   Comments (7)   |   Related: > Python


Problem

We may need to run a long-running application without human intervention. For example, we may want to monitor performance data on a schedule or react to specific events. The application should run in its session and not interfere with other applications. In this case, we can develop an application that runs as a Window Service (Altvater, 2017). Python applications have been proliferating for performing routine tasks in recent years. Therefore, IT professionals might want to run a Python application as a Windows service.

Solution

We can use the "Python for Windows Extensions" package (known as pywin32) to run a Python application as a Windows service. The pywin32 package allows us to use Win32 application programming interface (API) features on Python (PYPI, 2022). We often use the win32serviceutil module in this package to utilize APIs. For example, the module contains a base class "win32serviceutil.ServiceFramework" that helps create Windows services in Python (Robinson & Hammond, 2000). When creating a service in Python, we usually derive service classes from the base class. The service classes inherit methods and properties from the class. This approach simplifies working with Windows services.

We design a fictitious project to exhibit the process of creating a windows service in Python. MSSQLTips.com, owned by Edgewood Solutions, has served the SQL Server community since 2006. The website's landing page presents titles and short summaries of the most recent tips. In addition, every article comes with a featured image. Images have many purposes when used on web pages, besides helping the page look great (Simolo, 2016). However, for some reason, occasionally, some images may not display. Therefore, the CTO at Edgewood Solutions wants to monitor all featured images on the landing page to make sure all images display correctly.

We may have various Python applications on the same machine that depend on different versions of the same packages. In this case, each application should run in an isolated environment. Therefore, this project needs to run in a Python virtual environment.

We also need to use the "win32serviceutil" module to interact with Windows services. After briefly covering these prerequisites, we write an abstract class derived from the "win32serviceutil.ServiceFramework" class. We then create a service class that implements the abstract methods in order to perform the service functionality. Finally, the article explores a step-by-step process of creating a Windows service.

The author tested the project on Windows 10 Pro 10.0 <X64> and ran the Python scripts with Python 3.9.7 (64-bit). The IDE is Microsoft Visual Studio Community 2022 (64-bit).

1 – Prerequisite

There is a growing collection of several thousand third-party packages maintained by the Python community worldwide. Each package may have several versions. To avoid the version conflict, it is a good idea to run a Python application on a Python virtual environment. In addition, Python has complete support for Windows services, which are long-running applications in a Windows system.

1.1 Running a Python Script in a Virtual Environment

The tip "Creating a SQL Server Agent Job to Run Python Scripts in a Virtual Environment" demonstrates the steps to install Python on Windows. Without loss of generality, we can follow the instructions in the tip to create a virtual environment in the "C:\pyvirtuals\monitor_landing_page" folder. Click here to download the complete source code that contains the configuration file, "requirements.txt," for installing the virtual environment. We run the following commands to create the Python virtual environment for this project and install all necessary packages from the "requirements.txt" file:

 REM 1, Create a Virtual Environment
python -m venv C:\pyvirtuals\monitor_landing_page
 
REM 2, Activate the Virtual Environment
C:\pyvirtuals\monitor_landing_page\Scripts\activate.bat
 
REM 3, Install the Necessary Packages
pip install -r C:\pyapps\monitor_landing_page\requirements.txt

We copy all the Python scripts to the "C:\pyapps\monitor_landing_page" folder. The Python script "app.py" implements our business requirements. Before running the script as a Windows service, we manually run the script. We use the following command to run the Python script. Figure 1 illustrates the output of the program execution. We can also find the "log_file.txt" file in the "C:\pyapps\monitor_landing_page" folder. These outputs indicate that we run the Python script successfully.

 REM Run the Python script in the virtual environment.
python C:\pyapps\monitor_landing_page\app.py
python command and output

Figure 1 Running the Python Script in the Virtual Environment

1.2 Using the win32serviceutil Module to Connect to a Windows Service

The service control manager (SCM), started at system boot, maintains a database of installed and driver services. The SCM can control services by using control requests. For example, when a service needs to start, the SCM issues a control request to the service. The service acts on the request and reports its status to the SCM. In addition, the SCM provides a user interface to allow the user to start and stop services manually. The SCM also provides an API to enable programmatic control of services. This feature allows us to use a program to control services (Robinson & Hammond, 2000).

The Python win32serviceutil module provides some handy utilities to control existing Windows services. For example, we can connect a Windows service with some Python functions. We can conduct some tests to gain hands-on experience using this module. We start a Command Prompt as an administrator so that these Python functions have permission to access the services. The author's machine name is "myPy," and we test these functions against the "SQL Server VSS Writer" service. Here are the function syntaxes and Python statements used in this test:

 REM the function signature
StartService( serviceName , args=None , machine=None )
StopService( serviceName , machine=None )
QueryServiceStatus( serviceName , machine=None )
 
import win32serviceutil
win32serviceutil.StopService("SQL Server VSS Writer", "myPy")
win32serviceutil.StartService("SQL Server VSS Writer", None, "myPy")
win32serviceutil.QueryServiceStatus("SQL Server VSS Writer", "myPy")

Figure 2 demonstrates the steps to stop a Windows service. We use the "win32serviceutil.StopService" function to stop the service and the "win32serviceutil.QueryServiceStatus" function to check the service status. Both the functions return a SERVICE_STATUS structure. According to Microsoft Docs (MS Docs, 2021), the dwCurrentState "3" means that the service is stopping, and the dwCurrentState "1" suggests that the service is not running. The output indicates that we successfully stop the Windows service. Meanwhile, we can check the service status through the SCM interface. Figure 3 shows that the service stopped.

python command and output

Figure 2 Stopping the SQL Server VSS Writer Service on the Computer Named myPy

windows services

Figure 3 Check the Service Status through the SCM Interface

2 – Creating Subclasses of the win32serviceutil.ServiceFramework Class

Because of some special requirements for Windows service, we use the executable, PythonService.exe, to host Python services. When a Python service starts, the executable creates an instance of the Python service class and delegates all service functionality to this instance (Robinson & Hammond, 2000). The Python service class should provide methods to interact with the SCM. The module win32serviceutil offers a base class, "win32serviceutil.ServiceFramework," which includes some predefined methods, for example, ServiceCtrlHandler() and SvcRun(). Therefore, when writing a Python service, we often create a class inherited from the base class.

In practice, we often write an abstract service class that provides a blueprint for other classes. The abstract class provides a standard interface for different implementations of a component. This project creates an abstract class "PythonService" that inherits from the "win32serviceutil.ServiceFramework" class. This class is responsible for installing, debugging, starting, and stopping the service. We may share this class on different projects. Next, we write another class, "MointorImageService," that inherits from the "PythonService" class. We concentrate on the service functionality in this class.

2.1 Creating the Abstract Class to Interact with the SCM

The "win32serviceutil.ServiceFramework" class has already defined some methods. Most Python services are a subclass of this base class. Inspired by the smallest possible Python service written in Robinson & Hammond’s book, we create an abstract class to interact with the SCM. The abstract class declares the three abstract methods, i.e., start(), stop(), and main(), in which the derived class should have implementations. The following Python code exhibits the class, and the comments help explain the code.

# Reference:
# Mastromatteo, D. (2018): https://thepythoncorner.com/posts/2018-08-01-how-to-create-a-windows-service-in-python/
# Robinson, S. & Hammond, M. (2000). Python Programming on Win32. Sebastopol, CA: O'Reilly Media.
import win32event
import win32service
import win32serviceutil
from abcimport ABC,abstractmethod
class PythonService(win32serviceutil.ServiceFramework, ABC):
    @classmethod
    def parse_command_line(cls):
        ''' Parse the command line '''
        win32serviceutil.HandleCommandLine(cls)
    # Override the method in the subclass to do something just before the service is stopped.
    @abstractmethod
    def stop(self):
        pass
    # Override the method in the subclass to do something at the service initialization.
    @abstractmethod
    def start(self):
        pass
    # Override the method in the subclass to perform actual service task.
    @abstractmethod
    def main(self):
        pass
    def __init__(self, args):
        ''' Class constructor'''
        win32serviceutil.ServiceFramework.__init__(self, args)
        # Create an event which we will use to wait on.
        # The "service stop" request will set this event.
        self.hWaitStop = win32event.CreateEvent(None, 0, 0,None)
    def SvcStop(self):
        '''Called when the service is asked to stop'''
        # We may need to do something just before the service is stopped.
        self.stop()
        # Before we do anything, tell the SCM we are starting the stop process.
        self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
        # And set my event.
        win32event.SetEvent(self.hWaitStop)
    def SvcDoRun(self):
        '''Called when the service is asked to start. The method handles the service functionality.'''
        self.ReportServiceStatus(win32service.SERVICE_START_PENDING)
        # We may do something at the service initialization.
        self.start()
        self.ReportServiceStatus(win32service.SERVICE_RUNNING)
        # Starts a worker loop waiting either for work to do or a notification to stop, pause, etc.
        self.main()

2.2 Writing the Subclass to Perform Service Functionality

We create another new class that inherits methods and properties from the abstract class. The new class concentrates on the service functionality and some service configurations. Therefore, the new class should implement the three abstract methods declared in the abstract class. First, we set the running condition in the start() method. We then negate the running condition in the stop() method. Finally, the main() method executes a loop when the running condition is valid. The class also defines the service name and sets the location of the PythonService.exe. The following block demonstrates the code. It is worth noting that the code contains startup code that handles the command line when run as a script. We can install, debug, and start the service using the startup code.

# Reference:
# Mastromatteo, D. (2018): https://thepythoncorner.com/posts/2018-08-01-how-to-create-a-windows-service-in-python/
import app
import time
from PythonService import PythonService
class MointorImageService(PythonService):
    # Define the class variables
    _svc_name_ = "MointorImageOnLandingPageService"
    _svc_display_name_ = "MSSQLTips Mointor Images Service"
    _svc_description_ = "Mointor images on the landing page of the MSSQLTips.com."
    _exe_name_ = "C:\pyvirtuals\monitor_landing_page\Scripts\pythonservice.exe"
    # Override the method to set the running condition
    def start(self):
        self.isrunning =True
    # Override the method to invalidate the running condition
    # When the service is requested to be stopped.
    def stop(self):
        self.isrunning =False
    # Override the method to perform the service function
    def main(self):
        while self.isrunning:
            app.check_web_page_images()
            time.sleep(5)
# Use this condition to determine the execution context.
if __name__ == '__main__':
    # Handle the command line when run as a script
    MointorImageService.parse_command_line()

3 –Creating a Windows Service Using Python

We have already tested the app.py script in the virtual environment. We also created the abstract class "PythonService" and its subclass "MointorImageService." The "C:\pyapps\monitor_landing_page" folder should have these three files. All files from the virtual environment creation are in the "C:\pyvirtuals\monitor_landing_page" folder. We use this folder structure for demonstration purposes. In practice, we can use a different folder structure. The service account should have permission to write text into the log file. We then walk through a step-by-step process to run the Python scripts as a Windows service.

Step 1: Copy the executable, pythonservice.exe, to the virtual environment scripts folder.

Copy pythonservice.exe from the "C:\pyvirtuals\monitor_landing_page\Lib\site-packages\win32" folder to the "C:\pyvirtuals\monitor_landing_page\Scripts folder" (Lindsay, 2016). The service class variable "_exe_name" should contain the fully qualified path of the executable. The "MointorImageService" class definition demonstrates the class variable initialization. In practice, we often put the path into a configuration file.

Step 2: Open the command prompt as administrator.

Enter "command" in the Windows 10 search box. The "Command Prompt" appears in the pop-up list. Right-click on the "Command Prompt" item and select "Run as administrator" in the context menu (Glenn, 2021). We then change the directory to the "C:\pyapps\monitor_landing_page" folder.

Step 3: Run the pywin32_postinstall.py to register two dynamic link libraries.

The Python script pywin32_postinstall.py is in the "C:\pyvirtuals\monitor_landing_page\Scripts" folder. We first run the "C:\pyvirtuals\monitor_landing_page\Scripts\activate.bat" to activate the virtual environment. Then, we execute the Python script "C:\pyvirtuals\monitor_landing_page\Scripts\pywin32_postinstall.py" in the command window. Figure 4 presents the output of the two commands. Here are the two commands:

 C:\pyvirtuals\monitor_landing_page\Scripts\activate.bat
C:\pyvirtuals\monitor_landing_page\Scripts\pywin32_postinstall.py -install
python command and output

Figure 4 Execute the pywin32_postinstall.py in the command window

If we want to unregister these two dynamic link libraries, we can execute the following command to roll back the change:

 C:\pyvirtuals\monitor_landing_page\Scripts\pywin32_postinstall.py -remove

Step 4: Install the service

Execute the following command to install the service. Figure 5 demonstrates this step and presents the output. We can ignore the warning message because we have already run the pywin32_postinstall.py script. The "pythoncom39.dll" and "pywintypes39.dll" are in the "c:\Windows\system32" folder. The installation command also generates a Windows service log (Event ID 7045), as shown in Figure 6. The log briefly describes the newly installed service.

 python MointorImageService.py install
python command and output

Figure 5 Use the command prompt to install the service

event viewer log

Figure 6 The Window service log indicates the new installed service

Step 5: Start the service

There are several ways to start a service. A started service should report its status to the SCM and listen for control requests from the SCM. We can use the following command to start the service:

 python MointorImageService.py start

To ensure that the service starts correctly, we can check the log file, which should look like the following block.

 ***Start***: 20/06/2022 22:48:42
****End****: 20/06/2022 22:48:49
The execution time: 06.734439 seconds 
***Start***: 20/06/2022 22:48:54
****End****: 20/06/2022 22:48:59
The execution time: 05.593806 seconds

The SCM provides a user interface to control services, as shown in Figure 7. We can start the service process through the interface. By default, the service startup type is "manually," meaning a user should start the service manually. However, we can change the type to "automatically" so that the service automatically starts when the system boots. In addition, the service account is LocalSystem, a predefined local account used by the service control manager. The SCM allows us to change the service account.

windows services

Figure 7 Use the Service Control Manager to manually services

Step 6 (optional): Stop the service

The SCM provides a user interface to allow us to stop Windows services manually. When we stop a service, the SCM issues a control request to the service. The service then reacts to this request and reports its status to the SCM before it terminates. We also can use the following command to stop the service.

 python MointorImageService.py stop

Step 7 (optional): Debug the service

If the service does not work properly, we use the following command to execute the service in debug mode. We often use this command to test the service after installation. The logging module helps print the debug message to the console. Figure 8 shows the output of this command. We should turn off the console output when running the service on the production server.

 python MointorImageService.py debug
python command and output

Figure 8 Debug the Windows service

Step 8 (optional): Update the service

When changing the Python script, we can use the following command to update the service.

 python MointorImageService.py update

Step 9 (optional): Uninstall the service

We can also remove the service using a command line, as shown below:

 python MointorImageService.py remove

Summary

Running a Python application as a Windows service enables us to execute long-running Python applications that utilize Windows features. We can use the Service Control Manager (SCM) to control these services. The article created a fictitious project that extracts image information from the MSSQLTips' landing page. We wrote Python scripts to scrape the web page. We then ran the Python application as a Windows service.

We briefly covered creating a Python virtual environment and installing third-party packages. We then explored the "win32serviceutil" module and used functions to control Windows services with Python. Next, the article introduced the "win32serviceutil.ServiceFramework" base class. After creating the abstract class derived from the base class, we wrote the service class to implement the abstract methods. Finally, we walked through the step-by-step process of creating a Windows service.

Reference

Altvater, A. (2017). What are Windows Services? How Windows Services Work, Examples, Tutorials and More. https://stackify.com/what-are-windows-services/.

Glenn, W. (2021). How to Open the Command Prompt as Administrator in Windows 8 or 10. https://www.howtogeek.com/194041/how-to-open-the-command-prompt-as-administrator-in-windows-8.1/.

Lee, X. (2005). Python: List Modules, Search Path, Loaded Modules. http://xahlee.info/python/standard_modules.html.

Lindsay.stevens.au. (2016). Using PythonService.exe to host python service while using virtualenv. https://stackoverflow.com/questions/34696815/using-pythonservice-exe-to-host-python-service-while-using-virtualenv.

MS Docs. (2021). SERVICE_STATUS structure. https://docs.microsoft.com/en-us/windows/win32/api/winsvc/ns-winsvc-service_status.

PYPI. (2022). pywin32 304 - Python for Window Extensions. https://pypi.org/project/pywin32/.

Robinson, S. & Hammond, M. (2000). Python Programming On Win32. Sebastopol, CA: O'Reilly Media.

Simolo, G. (2016). When Words Meet Pictures: How to Use Text and Images to Create Striking Articles for Readers. https://www.freelancewriting.com/freelancing/when-words-meet-pictures/.

Next Steps


sql server categories

sql server webinars

subscribe to mssqltips

sql server tutorials

sql server white papers

next tip



About the author
MSSQLTips author Nai Biao Zhou Nai Biao Zhou is a Senior Software Developer with 20+ years of experience in software development, specializing in Data Warehousing, Business Intelligence, Data Mining and solution architecture design.

This author pledges the content of this article is based on professional experience and not AI generated.

View all my tips


Article Last Updated: 2022-07-11

Comments For This Article




Thursday, November 30, 2023 - 1:53:48 AM - Davood Back To Top (91782)
Hello Akrami,
I had the same problem.
error 0xC0000003 The instance's SvRun() method failed < Error getting traceback - traceback . print _ exception() failed > (null) : (null)
This problem is rising because there is a problem in our main code.
To determine your problem, change SvcDoRun() function of "pythonservice.py":
try:
self.main()
except Exception as e:
print (e)
os._exit(-1)
print(e) will show you your problem on main part of your code.
Note: to see the output of your print, run your service in debug mode.

Monday, May 8, 2023 - 8:10:15 PM - Nai Biao Zhou Back To Top (91171)
Hello Akrami,

Thank you for the question.

As I mentioned in the previous comment, the errors mostly come from the two dynamic link libraries. Indeed, there are many other possible reasons.

Could you please tell me the Python version and your OS? Then, if I can reproduce the error, it will be easy to figure it out.

Meanwhile, you may try an alternative solution: https://www.mssqltips.com/sqlservertip/7325/how-to-run-a-python-script-windows-service-nssm/.

Thanks!
Nai Biao Zhou

Friday, May 5, 2023 - 2:59:51 PM - Akrami Back To Top (91158)
Hello.
the service was not started automatically. and when i debug that, at the first it was run, but at the end the bellow error was shown.
error 0xC0000003 The instance's SvRun() method failed < Error getting traceback - traceback . print _ exception() failed > (null) : (null)

please help me , how to fix it?
thanks a lot.

Thursday, September 8, 2022 - 8:49:01 PM - Nai Biao Zhou Back To Top (90456)
Hello Nikolaj,

Mostly, the errors come from the two dynamic link libraries.

Could you please ensure the two DLLs are in the "c:\Windows\system32" folder?

In addition, could you please run the following command to debug the service?

python MointorImageService.py debug

The debug mode may give more informative error messages.
I noticed you used Python 3.10. Please ensure the "c:\Windows\system32" folder does not have other versions of the two DLLs.

In the worst scenario, you may try Python 3.9 to get the service to work first.

Thanks!
Nai Biao Zhou

Thursday, September 8, 2022 - 9:49:24 AM - Nikolaj NohrRasmussen Back To Top (90455)
Hi,
I have followed the very good description (as admin), but when I get to the final command: "python MointorImageService.py start" it gives me the error "Error starting service" due to a time-out of 30000ms (error 7009 followed by a 7000 in the system event log).
On top of what you describe, I have tried to:
1) Copy the two DLL's pywintypes310.dll and pythoncom310 to the .../Script folder.
2) Added these to the system PATH:
C:\pyvirtuals\...\Script
C:\pyvirtuals\...\site-packages\pythonwin\
C:\pyvirtuals\...\Lib\site-packages\pywin32_system32\

Any idea what could be the problem?
Thanks a lot in advance.

Thursday, July 14, 2022 - 7:57:14 PM - Nai Biao Zhou Back To Top (90264)
Hello Shivanshu,

Thank you for reading my tip. I am wondering if the function "win32serviceutil.InstallService" can help:

win32serviceutil.InstallService(
cls._svc_reg_clast_,
cls._svc_name_,
cls._svc_display_name_,
startType=win32service.SERVICE_AUTO_START
)

For more information, please see this page:
https://programtalk.com/vs4/python/13338/TrustRouter/client/usermode/winservice.py/

Wednesday, July 13, 2022 - 11:18:07 PM - Shivanshu Oliyhan Back To Top (90257)
Hi thanks for this article, is there a way to start the windows service automatically without using -auto install during installation. What I want is to setup the service installation to automatic on each boot.














get free sql tips
agree to terms