A smoke test software testing teams can use as a build gate checks a small set of critical behaviors before deeper testing starts. It should identify the build and environment, exercise a useful workflow, and record what passed or failed. A green health endpoint alone does not show that users can complete that workflow.
This guide uses a local Python service with five controlled outcomes. Three broken versions still return HTTP 200 from their health route. A separate probe catches those failures and distinguishes them from an environment it cannot reach.
The result is a narrow decision: proceed to the next testing stage or stop and investigate. It is not a release certificate. The example tests artificial records in memory, with no real accounts, database or external service.
A smoke test software testing teams can interpret
Readiness needs a stated scope
MDN defines a smoke test around critical software functions, checked before more detailed testing. The useful question is whether the build works well enough for the next planned tests to produce meaningful evidence.
For a record-management service, that might mean creating a record and reading its value back. For another product, the core path will differ. Choose it from what users need and what the next stage depends on.
A smoke suite is a sample. It should check a few key tasks well.
Each task needs a clear expected result. Adding every known edge case would give it a different job and a larger maintenance cost.
Set the stop rule before you run it. Otherwise, a team can reinterpret a failure as harmless simply because it wants to continue. Record who can investigate a failure and who owns a change to the rule.
Smoke, sanity, integration and regression
Teams use these names in different ways. Agree on their meaning in your test plan rather than assuming that a label defines the scope.
Smoke testing uses a small set of checks to judge readiness. A sanity check often means a focused check around a recent change. State which behavior that includes in your team.
Integration testing concerns interactions between parts. A smoke flow can cross several parts, so integration and smoke are not mutually exclusive categories. The same test can serve both purposes.
Regression testing checks for unwanted changes in behavior that should still work. Its depth follows the risks of the change. It is not a promise to test every possible input.
For smoke testing vs regression testing, decide what question each stage answers. A quick critical-path pass can allow a broader suite to start. It cannot replace the broader suite’s evidence.
Choose a small critical-path contract
Identify the thing being tested
Write down the intended build, environment and starting state. A response from yesterday’s service tells you little about today’s build. A test aimed at the wrong URL can be perfectly green and irrelevant.
In the example, the expected build is build-a. The service returns its build label from /health. The probe checks both that label and the HTTP status before it attempts a write.
That checks a reported label, not a signed artifact. A service could report a stale or incorrect label.
A real pipeline must link the built artifact to the instance under test. The test needs to check that link.
For example, record the artifact digest and the environment revision from the system that placed it there. Compare them with the target that the probe actually contacted. This is a proposed evidence practice, not something our local fixture proves.
Check the preconditions too. Is the test account permitted to act? Is the fixture data present, and can this run safely create and remove its own records?
A missing precondition should not become a silent skip. The owner needs to know whether the tested behavior failed or the test lacked the conditions needed to observe it.
Define an input and an expected result
Our small contract has four parts:
- The health response reports the expected build.
- Creating a record returns status 201 and an integer record ID.
- Reading that ID returns exactly the intended ID and name.
- Deleting the record returns status 204, and the local harness finds no stored record left behind.
The input name is smoke-fixture. The read-back check compares the whole expected object. A check that accepts status 200 but ignores the body would miss the wrong value below.
This is why serialization and round-trip checks matter. A request can succeed and return the wrong value. Choose assertions that cover the behavior your next stage needs.
Set a time budget for the chosen checks. Here, each HTTP connection has a two-second timeout, and the harness gives each child probe ten seconds. Neither value is a recommended duration for every smoke suite.
The connection timeout bounds a blocking socket call. It does not set a deadline for the whole suite. The child-process timeout adds a separate bound around this local probe.
A real runner needs a plan for a process that times out. Its work may still need cleanup.
One shaped part sits beside the recess it is meant to fit.
Run the gate against deliberately broken controls
The probe and its local fixture
Save the complete example below as smoke_fixture.py, then run python smoke_fixture.py. It uses Python’s standard library and loopback-only ports. The run reported here used CPython 3.12.14 on November 6, 2026.
Python’s ThreadingHTTPServer hosts the fixture, and HTTPConnection sends the requests. The server documentation warns against using this basic server in production. Here it exists only to make the controls repeatable.
The harness starts a fresh fixture for each reachable case. It runs the probe in a child process.
It checks that process’s actual exit code. It also inspects the fixture’s in-memory store after the probe finishes.
For the unreachable case, a socket reserves a local port without listening on it. That produces a controlled connection refusal. No real deployment is stopped or changed.
import http.client, json, socket, subprocess, sys, threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
class Failed(Exception):
pass
def check(condition, message):
if not condition:
raise Failed(message)
def request(port, method, path, payload=None):
conn = http.client.HTTPConnection('127.0.0.1', port, timeout=2)
try:
body = None if payload is None else json.dumps(payload)
conn.request(method, path, body, {'Content-Type': 'application/json'})
response = conn.getresponse()
raw = response.read()
return response.status, json.loads(raw) if raw else None
finally:
conn.close()
def probe(port):
report = {'checks': [], 'cleanup': 'not-needed'}
item_id = None
code = 0
try:
status, data = request(port, 'GET', '/health')
report['health_status'] = status
check(status == 200 and data.get('build') == 'build-a', 'wrong build or health response')
report['checks'].append('build-identity')
status, data = request(port, 'POST', '/items', {'name': 'smoke-fixture'})
check(status == 201 and isinstance(data.get('id'), int), 'create failed')
item_id = data['id']
report['checks'].append('create')
status, data = request(port, 'GET', f'/items/{item_id}')
check(status == 200 and data == {'id': item_id, 'name': 'smoke-fixture'}, 'read-back mismatch')
report['checks'].append('read-back')
except (ConnectionError, TimeoutError, OSError) as error:
code, report['error'] = 2, type(error).__name__
except (Failed, ValueError, AttributeError) as error:
code, report['error'] = 1, str(error)
finally:
if item_id is not None:
try:
status, _ = request(port, 'DELETE', f'/items/{item_id}')
check(status == 204, 'cleanup failed')
report['cleanup'] = 'passed'
except (Failed, ValueError, OSError) as error:
report['cleanup'] = 'failed: ' + type(error).__name__
code = code or 1
report['exit_code'] = code
return code, report
def fixture(mode):
records = {}
class Handler(BaseHTTPRequestHandler):
def log_message(self, *_):
pass
def send(self, status, value=None):
body = b'' if value is None else json.dumps(value).encode()
self.send_response(status)
self.send_header('Content-Type', 'application/json')
self.send_header('Content-Length', str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_GET(self):
if self.path == '/health':
self.send(200, {'build': 'build-old' if mode == 'wrong-build' else 'build-a'})
elif self.path.startswith('/items/'):
item_id = int(self.path.rsplit('/', 1)[1])
self.send(200, {'id': item_id, 'name': records[item_id]})
else:
self.send(404, {'error': 'not-found'})
def do_POST(self):
data = json.loads(self.rfile.read(int(self.headers['Content-Length'])))
if self.path != '/items':
self.send(404, {'error': 'not-found'})
elif mode == 'core-503':
self.send(503, {'error': 'unavailable'})
else:
records[1] = 'wrong-value' if mode == 'wrong-value' else data['name']
self.send(201, {'id': 1})
def do_DELETE(self):
records.pop(int(self.path.rsplit('/', 1)[1]), None)
self.send(204)
server = ThreadingHTTPServer(('127.0.0.1', 0), Handler)
thread = threading.Thread(target=server.serve_forever, kwargs={'poll_interval': .01}, daemon=True)
thread.start()
return server, thread, records
if __name__ == '__main__':
if len(sys.argv) > 1 and sys.argv[1] == '--probe':
code, report = probe(int(sys.argv[2]))
print(json.dumps(report, sort_keys=True))
sys.exit(code)
output = []
for mode, expected in [('healthy', 0), ('core-503', 1), ('wrong-value', 1), ('wrong-build', 1)]:
server, thread, records = fixture(mode)
try:
completed = subprocess.run([sys.executable, __file__, '--probe', str(server.server_port)],
capture_output=True, text=True, timeout=10)
report = json.loads(completed.stdout)
check(completed.returncode == expected, 'wrong process exit code')
check(report['health_status'] == 200, 'negative control must keep health green')
check(not records, 'fixture records were not cleaned up')
output.append({'mode': mode, **report, 'remaining_records': len(records)})
finally:
server.shutdown()
server.server_close()
thread.join()
with socket.socket() as reserved:
reserved.bind(('127.0.0.1', 0)) # Bound but not listening: controlled connection refusal.
completed = subprocess.run([sys.executable, __file__, '--probe', str(reserved.getsockname()[1])],
capture_output=True, text=True, timeout=10)
check(completed.returncode == 2, 'unreachable environment must not pass')
output.append({'mode': 'unreachable', **json.loads(completed.stdout)})
print(json.dumps({'python': sys.version.split()[0], 'results': output}, indent=2))A green health endpoint is insufficient
The observed results were:
| Fixture | Health status | Successful checks | Cleanup | Probe exit |
|---|---|---|---|---|
| Healthy | 200 | Build, create, read-back | Passed | 0 |
| Create returns 503 | 200 | Build | Not needed | 1 |
| Read returns wrong value | 200 | Build, create | Passed | 1 |
| Wrong build label | 200 | None | Not needed | 1 |
| Unreachable | No response | None | Not needed | 2 |
The healthy case passes when all required checks and cleanup finish. The harness also confirms that zero records remain. That proves the stated behavior for this fixture and this run.
The 503 case has a working health route but a failed create operation. The probe stops at that mismatch. No record was created by this fixture, so cleanup is not needed.
The wrong-value case is more subtle. Both the write status and read status look good.
The returned name is wrong, so the value check fails. The probe still deletes the known record.
The wrong-build case stops before any write. That avoids spending more time testing the wrong reported version. It does not explain why that version answered.
All four reachable fixtures returned health status 200. All had zero records left afterward.
The status code alone cannot tell these cases apart. The probe checks the build and the returned data too.
The right prism has a visible defect beside the intact one.
Know what the cleanup proves
The probe learns an ID from a successful create response. It uses that ID in a finally block to attempt deletion, even when read-back fails. If cleanup fails, the run cannot pass.
There is an important gap: a real server could create data and lose its response. Without a returned ID, this probe cannot find that data to remove it. The example does not implement that failure case or a recovery mechanism.
A real test may need a unique run marker or a safe way to find its data. Expiry is another choice.
Design this with the service owner. Do not delete records merely because their names resemble test data.
The sample uses one fixed ID and one probe at a time. It does not make writes safe to retry. The separate guide to idempotency and API retries covers why retries need an explicit contract.
Not every bad response or fault maps to a neat report state here. If the probe crashes or the harness times out, stop and find out why. Never infer a pass from missing output.
The screwdriver remains outside its empty place in the tray.
Turn results into the next testing decision
Pass, failed check or blocked evidence
Here, exit 0 means pass. Exit 1 means a required check failed; exit 2 means a connection blocked the check.
Those numbers are a local choice. State what your own runner’s codes mean.
A failure means an observed result did not meet the contract. It does not reveal the cause by itself. The wrong-value result could point to many different faults in a real application.
A blocked result means the probe could not get the response it needs. The product code or its setup might still cause a refused connection. Calling it blocked does not excuse it or establish that the application is healthy.
The rule proposed here stops the next stage for either result. Triage decides what happened and what to try next.
Link a later pass to the failed attempt. Note what changed between runs.
Consider the wrong-build result in a shared test environment. First keep the failed report. Check whether the test used the expected address and whether that address points to the intended build.
If the address is wrong, fix the target and run the full contract again. If the service reports an old build, ask the owner to check what is running there. Do not change the expected label just to make the check pass.
These are triage steps, not findings from the local run. The fixture tells us only that the labels differ. It cannot tell us who put a real build in place or why.
Do not hide repeated failures behind an unqualified retry. If a check is flaky, keep its attempts.
Find out why the results vary. A retry policy can change what a green result means.
Preserve the actual run record
Keep the expected and observed build identifiers, environment address or safe identifier, test version, start time and outcome. Record the failed check and cleanup result. Link that record to the later tests it permitted.
Exclude credentials and sensitive response data from reports. Keep enough evidence to explain a mismatch.
Avoid copying private records into test logs. Fake test data makes that boundary easier to manage.
Our JSON report includes checks, health status, errors, cleanup and exit code. It lacks a full record of the build and environment. Add that information in a real runner; do not claim the teaching report already contains it.
A check’s success must also control the next stage. GitHub’s workflow syntax documents that jobs using needs normally require their dependencies to succeed.
If a needed job fails or is skipped, the later job is skipped too. A condition can change that rule.
For example, always() can allow a dependent job to run despite an earlier failure. That can help a report run after a failure.
But a report and a gate have different jobs. Review the actual conditions and error-handling settings.
No GitHub workflow was executed for this example. The exit codes do not prove that your repository blocks a later stage.
They do not prove that it requires a check or protects a branch. Test those connections separately before relying on them.
Manual versus automated smoke testing
A person can follow a small smoke contract. They can record the build and the actual results.
That can help when the important behavior requires human judgment or the workflow is still changing. The checklist should still state inputs, expected behavior and the stop rule.
Automated checks help when the same test must run often. The result must be clear enough for code to check.
That code and data need upkeep too. A test that keeps clicking an obsolete control can waste time or observe the wrong behavior.
Browser checks add a view that this HTTP fixture lacks. They can interact with rendered controls and examine the resulting page. They need their own setup, assertions and failure evidence.
The supplementary Cypress.io workshop, Instant Smoke Testing with Cypress, features Natalie Lunbeck and offers that browser-testing perspective. It uses a different stack. It is not evidence that this Python probe checks browser behavior, and no Cypress commands from it are reproduced here.
Choose manual, automated or mixed checks to suit the task. Weigh their upkeep cost against the evidence they give.
Avoid a universal case count or runtime promise. A slow, unreliable gate and a fast, meaningless gate both deserve revision.
When an environment changes, first check whether the contract is still valid. Update expected behavior deliberately. Do not relax an assertion solely to restore a green badge.
Keep the sample and the policy current
Keep versions of the probe, its input data and expected results. If the core workflow changes, review the smoke sample as part of that change. A preserved test can still become irrelevant.
For example, an HTTP response might only acknowledge a queued job. Reading a local object immediately afterward would not prove that the downstream work completed. The distinction between webhooks and APIs helps identify which event you actually need to observe.
Our fixture has no asynchronous job or external dependency. Its data lives in a dictionary in one process. It does not validate persistence, authentication, permissions, TLS, browser rendering or a real deployment.
It also sends the core requests one at a time. A successful run says nothing about overlapping requests. Use a separate test, such as the race-condition reproduction, when concurrency is the question.
Nor is one build check a substitute for health monitoring and actionable alerts. Monitoring follows a running service over time. This smoke probe samples a stated behavior at one point in a test sequence.
Keep the negative controls as well as the passing case. If a future edit makes the wrong-value fixture pass, the gate has lost an assertion that mattered. A passing healthy fixture alone would not reveal that loss.
These controls check how this probe behaves under fixed conditions. They are not a measured failure rate, a performance study or proof that all possible defects are detected. Add controls when a concrete new failure changes what the gate must distinguish.
Let the evidence set the next step
Before using a smoke gate, name the build and environment, choose the smallest useful workflow, and state its expected results. Include cleanup and a clear owner for failed or blocked runs. Check that the runner uses the result to start or stop the next stage.
Then report what was observed. Our local healthy fixture passed its stated contract; the three green-health failures did not, and the unreachable case stayed blocked. Broader tests and release requirements still have work to do.