- October 19, 2026
- Posted by: editor
- Category: RECD & Emission Control Devices
A month of Retrofit Emission Control Device data looks clean on the dashboard. The differential pressure trend is smooth, there are no alarms, and back-pressure stays well below the limit. Then someone notices that the generator ran for 62 hours that month, but the RECD records only cover 41 of them. A third of the device’s operating time is simply missing, and nobody can say what happened during it.
Missing data is easy to overlook, because an absence doesn’t look like a problem. A trend line drawn across a gap looks continuous. A frozen value looks steady. A dashboard with no alarms looks healthy. Yet missing or invalid data undermines everything built on it: maintenance decisions, alarm reliability, compliance evidence and estimates of particulate avoided.
This guide explains the forms missing data takes in RECD monitoring, how to detect each one, how to calculate data availability properly, and what to do when gaps are found, including what not to do.
Why Missing Data Matters for an RECD
- Monitoring is only as good as its coverage. If the RECD wasn’t monitored during a long outage, a problem during that run could go unnoticed.
- Trends become misleading. A gap during a period of rising ΔP can hide the moment the device started to block or crack.
- Alarms can’t fire without data. If values aren’t arriving, the alarm system is blind. See how to design an RECD alarm response matrix for operators.
- Reports overstate certainty. Monthly KPIs calculated only from available data can look better than reality.
- Compliance evidence weakens. An inspector or auditor reviewing records will notice large unexplained gaps.
In regulated industrial sectors, continuous emission monitoring systems are subject to formal data availability expectations. CPCB’s guidelines on continuous emission monitoring systems are one example, and independent reviews such as a CEEW study of state OCEMS portals have highlighted how often data gaps occur in practice. Those rules apply to designated industrial stacks, not to DG set RECDs, but the principle is the same: data you can’t account for is evidence you can’t use.
The Forms Missing Data Takes
“Missing data” isn’t only blank rows. Watch for all of these:
| Type | What it looks like | Typical causes |
|---|---|---|
| Gaps | No records for a period | Communication loss; power loss to controller or gateway; logger full; platform outage |
| Frozen (flatlined) values | The same value repeated exactly for a long time | Controller frozen; gateway repeating last value; sensor stuck; BMS holding last value after comms loss |
| Out-of-range values | Physically impossible readings (e.g. ΔP of 6,553.5 mbar, temperature of −999) | Sensor open circuit; error codes read as values; scaling errors |
| Implausible values | Within range but inconsistent with operating state (e.g. high ΔP while DG stopped) | Wrong channel mapping; timestamp misalignment |
| Duplicated records | The same timestamp appears twice, often with different values | Re-uploads after buffering; clock corrections |
| Silent backfill | Data arrives later with arrival-time stamps instead of measurement times | Gateway buffering; platform settings |
| Partial records | Some fields present, others empty | One sensor faulty; partial register reads |
Some of these are true absences. Others are invalid data that looks present. Both count as missing for analysis purposes.
Step 1: Define When Data Should Exist
You can’t detect missing data without knowing what “complete” looks like. For an RECD, data matters most when the DG is running. Readings while the engine is stopped are less important, and a ΔP of zero then is expected, not missing.
So define:
- Expected logging interval, for example one sample per minute
- The period that matters: DG running time, taken from the DG controller’s start and stop records
- Expected samples = DG running minutes ÷ logging interval
To do this, the RECD data and DG runtime records must be on the same timeline. See how to align RECD sensor timestamps with generator runtime logs.
Step 2: Calculate Data Availability
The core metric is:
Data availability (%) = Valid samples received during DG running time ÷ Expected samples during DG running time × 100
“Valid” means present, within range, not frozen and consistent with operating state.
Worked example
The figures are illustrative, for one DG set over one month with one-minute logging:
- DG running time from the DG controller: 42 h 30 min = 2,550 minutes
- Expected samples: 2,550
- Samples received during running time: 2,247
- Of those, flagged invalid:
- 41 frozen (identical ΔP for over 30 minutes during one run while load changed)
- 22 out of range (outlet temperature sensor open circuit for 22 minutes)
- Valid samples: 2,247 − 41 − 22 = 2,184
Data availability = 2,184 ÷ 2,550 × 100 = 85.6%
Breaking down the 366 missing or invalid minutes:
| Cause | Minutes | Share of missing |
|---|---|---|
| Gateway lost mobile signal during 12 Oct outage | 180 | 49% |
| Controller on grid-only supply; blind at start of 19 Oct outage | 120 | 33% |
| Frozen values (gateway repeating last value) | 41 | 11% |
| Outlet temperature sensor fault | 22 | 6% |
| Scattered single-sample drops | 3 | 1% |
| Total | 366 | 100% |
The breakdown shows that 82% of the missing data came from two fixable infrastructure problems: weak signal and an unbacked power supply. See RECD monitoring during a power failure: what should stay available.
Why not calculate availability over the whole month?
If you divide by every minute in the month (43,200), including all the time the DG was stopped, availability looks excellent even if the device was barely monitored while running. Always calculate against DG running time, because that’s when the RECD is working and the data matters.
Step 3: Detection Techniques
Gap scanning
Sort records by measurement timestamp and look for intervals longer than the expected logging interval plus a tolerance. For one-minute logging, any interval over, say, 2 minutes is a gap. Most spreadsheets and BMS tools can do this. Summarise the gaps by start, end and duration.
Cross-checking against DG runtime
For every DG run in the DG controller log, check that RECD data exists from just after start to just before stop. A run with no RECD data, or data only for part of it, stands out immediately.
Heartbeat and communication status
If the controller provides a heartbeat counter (a value that increments regularly), a counter that stops changing means the controller is frozen, even if values are still arriving. Communication status flags from the BMS or gateway show when the link was down. See RECD Modbus data: units, scaling and register mapping explained.
Flatline (frozen value) detection
Real sensor readings fluctuate slightly. Flag any value that stays exactly the same for longer than a set time while the DG is running, for example 15 minutes, especially if load or other values changed during that time. Apply this to analogue values such as ΔP, back-pressure and temperatures.
Range checks
Set physical limits for each value: ΔP between 0 and the sensor’s full scale, temperatures within a sensible range, back-pressure non-negative. Values outside these limits are invalid, often error codes or open-circuit readings.
Consistency checks
Check that values make sense together:
- ΔP and back-pressure should be near zero when the DG is stopped, and above zero when it runs
- Inlet temperature should rise after a DG start
- Outlet temperature shouldn’t be far above inlet temperature except during regeneration
- ΔP should generally rise with load
Inconsistent combinations suggest a wrong channel, a timestamp offset or a faulty sensor.
Duplicate and backfill checks
Look for repeated timestamps, and for bursts of records arriving at the same arrival time but carrying different measurement times. These show buffered uploads, which are fine if measurement times are kept.
How Missing Data Biases Results
Gaps don’t just reduce the amount of data. They often skew what remains, because data tends to go missing under particular conditions, such as long outages and high loads.
Worked example: a misleading average
The figures are illustrative. In one month, a DG set had:
- 10 weekly test runs at about 20% load, totalling 150 minutes, with average ΔP of 25 mbar. All data was captured, because the grid was on and the gateway was powered.
- 2 long outage runs at about 70% load, totalling 600 minutes, with average ΔP of 60 mbar. Only 200 minutes were captured, because the gateway was on a grid-only supply and lost power partway through.
Average ΔP from the captured data only:
(150 × 25 + 200 × 60) ÷ (150 + 200) = (3,750 + 12,000) ÷ 350 = 45.0 mbar
True time-weighted average ΔP over all running time:
(150 × 25 + 600 × 60) ÷ (150 + 600) = (3,750 + 36,000) ÷ 750 = 53.0 mbar
The captured data understates the device’s typical loading by about 15%, because the missing data was concentrated in the hardest-working periods. The same bias affects peak back-pressure, regeneration counts and any running-hours figure taken from RECD data instead of the DG controller.
The lesson: always report which conditions the gaps occurred in, not just how many minutes are missing. A small gap during a heavy outage can matter more than a large gap during idle test runs.
Step 4: Flag, Don’t Fabricate
When gaps are found, the most important rule is: mark them clearly and explain them, don’t fill them with invented numbers.
- Flag each missing or invalid period in the data with a reason code, such as COMMS, POWER, SENSOR, FROZEN or RANGE
- Show gaps as gaps on trend charts, not as straight lines joining the last and next values
- Record the cause and fix in the event log. See what should an RECD event log record
- Report availability alongside any KPI calculated from the data
Is interpolation ever acceptable?
For visual smoothing of very short gaps, such as one or two missing samples on a trend chart, interpolation is harmless if it’s labelled. It’s not acceptable to fill gaps to make records look complete for compliance, warranty or reporting purposes. Any estimates used in calculations, such as running hours for a PM-avoided figure, should come from reliable sources like the DG controller, and be clearly identified. See estimating kilograms of PM avoided after an RECD retrofit.
Step 5: Fix the Causes
Detection is only useful if it leads to fixes. Common causes and remedies:
| Cause | Remedy |
|---|---|
| Controller or gateway on grid-only supply | Move to DG-backed essential supply with UPS or battery backup |
| Weak mobile signal | External antenna; dual-SIM gateway; relocate gateway |
| Gateway or BMS holds last value after comms loss | Configure to mark data as stale; add heartbeat and comms-fail alarms |
| Controller memory full; oldest data overwritten | Export more often; increase logging interval for less critical points; upgrade logging |
| Sensor faults from heat or vibration | Repair or replace; re-route cables; see RECD cable routing near hot exhaust lines |
| Platform stamps arrival time | Configure measurement-time stamping |
| Clock reset after power loss | Replace backup battery; enable time sync |
| Configuration change broke a mapping | Change control; re-test mappings after firmware updates |
Step 6: Set a Target and Review Monthly
Set a data availability target during DG running time that fits your site’s importance. For example, at least 95% for critical sites, and at least 90% for others. Then:
- Calculate availability every month, per DG set
- Break down missing time by cause
- Compare against the target
- Assign fixes for the biggest causes
- Track availability over time on the management dashboard. See what should a useful RECD operating dashboard show
A falling trend in availability is an early warning that monitoring infrastructure is deteriorating, whether from ageing batteries, failing sensors or a changed network.
Real-Time Detection: Don’t Wait for the Monthly Report
Monthly review catches patterns, but some gaps need attention immediately, especially during a DG run. Set up real-time alerts for:
- “DG running, no RECD data for X minutes”, the most important one
- Heartbeat stopped, meaning the controller is frozen
- Communication failure, meaning the link is down
- Sensor fault, meaning an out-of-range or open-circuit reading
At unattended sites, these alerts are essential. See how to set up remote RECD alerts for an unattended site.
Building Detection Into the Dashboard
Data quality shouldn’t only appear in a monthly report. Make it visible where people already look:
- A freshness indicator on every live page (“Updated 6 s ago” or “STALE”)
- Gaps shown as breaks in trend lines, not joined up
- Shaded periods on trends where data was invalid, with the reason on hover or click
- A data availability tile on the management view, by DG set and month
- A portfolio column showing availability for each site, so weak sites stand out
See what should a useful RECD operating dashboard show.
Questions to Ask Your Monitoring Provider
- How does the system mark data as stale or invalid, rather than repeating last values?
- Does it keep measurement timestamps through buffering and upload?
- How long can the gateway buffer data during a connection loss?
- Can it calculate data availability against DG running time automatically?
- Are flatline and range checks available and configurable?
- Can invalid periods be flagged with a reason, and are those flags exported with the data?
A Simple Data Quality Report Template
| Field | Value |
|---|---|
| DG set / RECD ID | |
| Period | |
| DG running time (from DG controller) | |
| Expected samples | |
| Valid samples | |
| Data availability | |
| Missing time by cause (comms / power / sensor / frozen / range / other) | |
| Longest single gap (start, end, cause) | |
| Runs with no RECD data at all | |
| Actions taken / planned | |
| Reviewed by, date |
Keep these reports with the RECD’s records. They show that monitoring is checked and maintained, which matters during a Pollution Control Board inspection or a warranty discussion.
Special Cases
Short DG runs
Weekly test runs of 10–15 minutes give few samples. One missing sample in a 10-minute run is 10% of that run’s data. Look at availability over the whole month rather than run by run, but still check that each run has some data.
Very long outages
During multi-day outages, controller memory may fill and mobile networks may be congested. Plan export routines and buffer sizes for the longest outages your site has experienced, not the average one.
Passive RECDs with manual readings
If your RECD has no electronic monitoring, the same principle applies to manual logs. Check that routine readings were actually taken as scheduled, and record missed checks honestly. See why RECD test results may differ from day-to-day site readings.
Related Reading
- How to set up remote RECD alerts for an unattended site
- What should a useful RECD operating dashboard show?
- What should an RECD event log record?
- How to align RECD sensor timestamps with generator runtime logs
- Integrating RECD monitoring with a building management system
Frequently Asked Questions
What’s a reasonable data availability target for RECD monitoring?
It depends on how critical the site is. A target of at least 90–95% of DG running time is a sensible starting point, with higher targets for hospitals, data centres and other critical sites. What matters most is measuring it honestly and fixing the main causes of gaps.
Should I count data when the DG is stopped?
Not for availability. RECD readings matter while the engine runs. Calculating availability over the whole calendar inflates the figure and hides gaps during actual operation.
How can I tell a frozen value from a genuinely steady reading?
Real sensor readings fluctuate slightly, even under steady conditions. A value that stays exactly the same to the last digit for a long period, especially while load or other readings change, is likely frozen. A heartbeat counter confirms whether the controller is still updating.
Can I fill gaps with estimated values for my reports?
Not for compliance, warranty or performance records. Gaps should be flagged and explained. For visual charts, very short gaps can be smoothed if clearly labelled. For calculations, use reliable sources such as DG controller running hours, and state any assumptions.
Where should DG running time come from for the availability calculation?
From the DG controller’s own start/stop records or hour meter, not from the RECD data. If running time is taken from the RECD data, any period with no RECD data also disappears from the running time, and availability looks artificially perfect.
How often should we check for missing data?
Use real-time alerts for gaps during DG runs, and review data availability monthly. After any long outage, check that day’s data specifically, because outages are when gaps are most likely.
What’s the most common cause of missing RECD data?
Monitoring equipment powered only from the grid supply, so it goes blind when the DG runs during an outage. Weak mobile signal at gateways and BMS links that hold the last value instead of flagging a failure are also common.
Know What You Don’t Know
Monitoring records are only trustworthy when you know where they’re incomplete. Aceget supplies CPCB type-approved Retrofit Emission Control Devices for DG sets from 25 kVA to 2500 kVA. We can help you design monitoring that keeps gaps to a minimum and makes them visible when they happen. See our certifications, or contact our team.