Python programme executes all lines but does not run any 'subprocess.Popen()' lines

I have simple Firefox add-on that, I am using it to test and discover the capabilities of Native Messaging, something I recently came across.

The add-on side of things works as I expect it, nothing out of order there, I am struggling with the Python side of the add-on.

The add-on works like this:

  1. On the browser, when the page HTML basics is visited
  2. The add-on will send the message “ping” to a Python script, on the local system, via stdin
  3. The python script should reply back to the browser the message it received via stdin and run a process via subprocess.Popen()

Sure it enough, in all my tests, on the browser console, I can see the Python programme sending back a reply like THIS. But the line subprocess.Popen(["explorer", "C:/Temp"]) is never executed at all. No matter where I place it in the Python script.

If I create a separate Python script with just the following code and run it by double clicking the file in explorer, it works. A explorer window is created:

import subprocess
subprocess.Popen(["explorer", "C:/Temp"])

Of course I am looking to do more than just open a explorer window, its just a simple example. The point is for some reason, my Python programme is stuck either reading at stdin or somewhere else.

I tried restructuring the Python code to something simple and tried “closing” the stdin stream to see if that will help it carry on with the execution of the remaining lines:

import sys
import json
import struct
import subprocess

rawLength = sys.stdin.buffer.read(4)
if len(rawLength) == 0:
	sys.exit(0)
messageLength = struct.unpack('@I', rawLength)[0]
message = sys.stdin.buffer.read(messageLength).decode('utf-8')
sys.stdin.buffer.flush()   						#Try closing the stdin buffer
sys.stdin.buffer.close()   						#Try closing the stdin buffer

subprocess.Popen(["explorer", "C:/Temp"])    #Again not executed

Same issue persists, the last line is again not executed. I am new to Python, JavaScript and add-on development. I asked around for any debugging tools for such a novel, edge use case but sadly I have not turned up with answers. The python programme does not spawn its own console window so its hard to tell where execution is stuck at with something like print()

I did try the following in its own python script file:

import sys
import json
import struct
import subprocess

rawLength = sys.stdin.buffer.read(4)
print(rawLength)
subprocess.Popen(["explorer", "C:/Temp"])

It will spawn its own console window, the programme is stuck at rawLength = sys.stdin.buffer.read(4) and will remain there, even if I provide just a letter and press enter, it continues when I provide four letters, opening file explorer at c:/Temp.

Last time I asked around. I was told this might be what is happening and I looked for a way to stop the stdin stream reading or close it, which is what I tried to do with flush()/close() but it does not help.

Am I attempting to close the stdin stream the right way? If so am I succeeding? How does one know for sure? Is stdin even the culprit here?

I am out of ideas, any help would be greatly appreciated!


For completeness, my add-on is compromised of only two files, a manifest.json and a background.file.

Manifest.json file:

{
"name": "test",
"manifest_version": 2,
"version": "1.0",

"browser_action": {"default_icon": "icons/message.svg"},
"browser_specific_settings": {"gecko": {"id": "test@example.org","strict_min_version": "50.0"}},

"background": {"scripts": ["background.js"]},
"permissions": ["tabs","activeTab", "webRequest", "<all_urls>", "nativeMessaging"]
}

Background.json file:

browser.webRequest.onCompleted.addListener(sendNativeMsg, {urls:["https://developer.mozilla.org/en-US/docs/Learn/Getting_started_with_the_web/HTML_basics"]}); 
function onResponse(response) {console.log(`INCOMING MSG: ${response}`);}

function sendNativeMsg(activeTab) {
	let thisMsg = "ping"
	console.log(`OUTGOING MSG: "${thisMsg}"`);	

	let sending = browser.runtime.sendNativeMessage("test", thisMsg);
	sending.then(onResponse);
	}

And the source code for the Python script is the following, which I got from the Native Messaging, MDN page, linked above:

import sys
import json
import struct
import subprocess

# Read a message from stdin and decode it.
def getMessage():
	rawLength = sys.stdin.buffer.read(4)
	if len(rawLength) == 0:
		sys.exit(0)
	messageLength = struct.unpack('@I', rawLength)[0]
	message = sys.stdin.buffer.read(messageLength).decode('utf-8')
	return json.loads(message)

# Encode a message for transmission,
def encodeMessage(messageContent):
	encodedContent = json.dumps(messageContent, separators=(',', ':')).encode('utf-8')
	encodedLength = struct.pack('@I', len(encodedContent))
	return {'length': encodedLength, 'content': encodedContent}

# Send an encoded message to stdout
def sendMessage(encodedMessage):
	sys.stdout.buffer.write(encodedMessage['length'])
	sys.stdout.buffer.write(encodedMessage['content'])
	sys.stdout.buffer.flush()

while True:
	subprocess.Popen(["explorer", "C:/Temp"])       #This line is never executed. The lines after here are executed.
	receivedMessage = getMessage()
	if receivedMessage == "ping":
		sendMessage(encodeMessage('stdin was "' + receivedMessage + '", Task is done'))

When the browser sends the “ping” message, the Python script is supposed to reply and execute the subprocess.Popen() line. However, it looks like the execution of subprocess.Popen() is happening before the reply is sent.

while True:
    receivedMessage = getMessage()
    if receivedMessage == "ping":
        sendMessage(encodeMessage('stdin was "' + receivedMessage + '", Task is done'))
        subprocess.Popen(["explorer", "C:/Temp"])

the subprocess.Popen() line will be executed only if the received message is “ping”.

Sharp eye for detail :slight_smile:
I tried moving that that line (subprocess.Popen()) around everywhere, in and out of indentation blocks ets, it does not run.

Just to make sure I was not going crazy I tried your suggestion, still nothing.

I appreciate the extra pair of eyes though. :+1:

you can try using subprocess.run instead of subprocess.Popen and add stdout=subprocess.PIPE to capture the output.

while True:
    process = subprocess.run(["explorer", "C:/Temp"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    process.wait()  # Wait for the process to complete
    receivedMessage = getMessage()
    if receivedMessage == "ping":
        sendMessage(encodeMessage('stdin was "' + receivedMessage + '", Task is done'))

the script waits for the explorer process to complete before moving on to the next line of code.

Add error handling to see if there are any issues with the subprocess call. You can capture the output and error streams to investigate further.

try:
    subprocess.Popen(["explorer", "C:/Temp"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
except Exception as e:
    print(f"Error executing subprocess: {e}")

Amazing, that did it. Its surreal!

I had no idea about run() but it seems that is what I needed.

As you mentioned, I noticed that on the browser console it does not receive back any message from the Python programme, is it by any chance possible to get the lines after subprocess.run(["explorer", "C:/Temp"] to not wait for the process to exit or finish?

I tried experimenting with your solution from the above to something like:

while True:
	process = subprocess.run(['pwsh', 'C:/Temp/testProg.ps1'])
	receivedMessage = getMessage()
	if receivedMessage == "ping":
		sendMessage(encodeMessage('stdin was "' + receivedMessage + '", Task is done'))

But still the browser console prints out an error saying it did not get a message back. If its not possible with run() its fine with me. I am more than happy with the solution and can carry on with it.

Anyways, thank you for the help. I am so glad to conclude this chapter!

1 Like

glad to help. Heppy coding!..

1 Like