Content
# Re: FW:
> **Thread Summary:** The email thread discusses an alarm anomaly involving a delayed callout response for an Influent Pumps UPS Trip alarm that occurred on April 20, 2026. Mason Radke investigated and found that the alarm's rapid on/off "chattering" caused the notification system (WIN-911) to skip alerts due to its configuration, which was then updated to notify on both active and inactive unacknowledged states to prevent future missed calls. Mychal Jones acknowledged the report and proposed coordinating on-site troubleshooting during an upcoming SCADA server upgrade to address the root cause of the alarm chatter.
---
## 1. From: Kevin Seifert <kevin@autosysnet.com> — Wed, 29 Apr 2026 03:10:21 +0000
@Mason Radke<mailto:mason@autosysnet.com>
Please look into this alarm anomaly.
-Kevin Seifert, AutoSys LLC
________________________________
From: Mike Arias <arias@sslocsd.us>
Sent: Tuesday, April 21, 2026 12:19 PM
To: Kevin Seifert <kevin@autosysnet.com>
Cc: Mychal Jones <mychal@sslocsd.us>
Subject: FW:
Good Afternoon Kevin,
So this alarm was received by Mychal Jones around 3:45am Monday. The call did not go to the On Call phone and it did not go to my phone, which it should have gone to both before going to Mychal. Both phones were checked and there were no missed calls. Let us know what you find out. Thank you.
Kind regards,
Michael Arias
Michael J. Arias
Operations Supervisor
Grade III Operator
South San Luis Obispo County Sanitation District
1600 Aloha Place, Oceano, Ca.
805-489-6666
[South San Luis Obispo County Sanitation District]
From: Mike Arias <mike_arias86@yahoo.com>
Sent: Tuesday, April 21, 2026 12:15 PM
To: Mike Arias <arias@sslocsd.us>
Subject:
[Photo attachment: 20260421_100131.jpg]
Yahoo Mail: Search, Organize, Conquer<https://mail.onelink.me/107872968?pid=nativeplacement&c=US_Acquisition_YMktg_315_SearchOrgConquer_EmailSignature&af_sub1=Acquisition&af_sub2=US_YMktg&af_sub3=&af_sub4=100002039&af_sub5=C01_Email_Static_&af_ios_store_cpp=0c38e4b0-a27e-40f9-a211-f4e2de32ab91&af_android_url=https://play.google.com/store/apps/details?id=com.yahoo.mobile.client.android.mail&listing=search_organize_conquer>
---
## 2. From: Mason Radke <mason.radke@gmail.com> — Wed, 29 Apr 2026 07:46:16 -0600
Investigated the delayed callout response for the Influent Pumps UPS Trip
alarm (UPS_0112_ON_BACKUP_Alm_TgtDisagree) that activated at 03:45:55 on
4/20/2026 and was not acknowledged until 04:03:37. Review of the WIN-911
logs showed that the alarm condition chattered repeatedly with very short
active durations (5–7 seconds per occurrence) before clearing, and the
assigned callout workflow ("Test 2") was configured to send notifications
only when the alarm was in the Active + Unacknowledged state. Because the
workflow's callout list cycles through contacts at one-minute intervals,
each dispatch attempt evaluated the alarm state at the moment of dispatch —
and since the alarm was inactive (but still unacknowledged) for the
majority of the cycle, WIN-911 correctly skipped notifications to the
District Call Out Phone, Mike Arias, and Mycal Jones contacts on multiple
passes per the configured Send Notification settings. A successful dispatch
eventually occurred to Mycal Jones at 03:54:58 when the workflow cycle
happened to coincide with a brief active window; the call connected on each
of four attempts but failed three times due to invalid authorization code
entry on the receiving end before being acknowledged on the fourth attempt
at 04:03:37. To prevent this filtering behavior from suppressing future
callouts on chattering alarms, the Test 2 workflow's Notifications Behavior
was updated to include both Active + Unacknowledged and Inactive +
Unacknowledged states, ensuring the callout chain proceeds to completion
based on acknowledgment status rather than instantaneous alarm state. It is
additionally recommended that the underlying chatter on the UPS
target/feedback disagreement signal be investigated at the source (PLC
logic or UPS feedback wiring) to address the root cause.
On Tue, Apr 28, 2026 at 9:10 PM Kevin Seifert <kevin@autosysnet.com> wrote:
> @Mason Radke <mason@autosysnet.com>
>
> Please look into this alarm anomaly.
[... earlier replies trimmed ...]
--
-Mason
---
## 3. From: Mychal Jones <mychal@sslocsd.us> — Wed, 29 Apr 2026 17:41:11 +0000
Mason,
Thank you for investigating and providing a comprehensive report on your findings.
Please let me know when we can coordinate troubleshooting the issue on site. We can probably coordinate it during the SCADA server upgrade site work.
Thank you,
![[20260429_19dda54bc8f3_image001.jpg]]
Mychal Jones
Plant Superintendent/Chief Plant Operator
Phone 805-489-6666 ext. 103
Web https://sslocsd.org/ Email mychal@sslocsd.us
1600 Aloha Place, Oceano, CA. 93445
From: Mason Radke <mason.radke@gmail.com>
Sent: Wednesday, April 29, 2026 6:46 AM
To: Kevin Seifert <kevin@autosysnet.com>
Cc: Mike Arias <arias@sslocsd.us>; Mason Radke <mason@autosysnet.com>; Mychal Jones <mychal@sslocsd.us>
Subject: Re: FW:
Investigated the delayed callout response for the Influent Pumps UPS Trip alarm (UPS_0112_ON_BACKUP_Alm_TgtDisagree) that activated at 03:45:55 on 4/20/2026 and was not acknowledged until 04:03:37. Review of the WIN-911 logs showed that the alarm condition chattered repeatedly with very short active durations (5–7 seconds per occurrence) before clearing, and the assigned callout workflow ("Test 2") was configured to send notifications only when the alarm was in the Active + Unacknowledged state. Because the workflow's callout list cycles through contacts at one-minute intervals, each dispatch attempt evaluated the alarm state at the moment of dispatch — and since the alarm was inactive (but still unacknowledged) for the majority of the cycle, WIN-911 correctly skipped notifications to the District Call Out Phone, Mike Arias, and Mycal Jones contacts on multiple passes per the configured Send Notification settings. A successful dispatch eventually occurred to Mycal Jones at 03:54:58 when the workflow cycle happened to coincide with a brief active window; the call connected on each of four attempts but failed three times due to invalid authorization code entry on the receiving end before being acknowledged on the fourth attempt at 04:03:37. To prevent this filtering behavior from suppressing future callouts on chattering alarms, the Test 2 workflow's Notifications Behavior was updated to include both Active + Unacknowledged and Inactive + Unacknowledged states, ensuring the callout chain proceeds to completion based on acknowledgment status rather than instantaneous alarm state. It is additionally recommended that the underlying chatter on the UPS target/feedback disagreement signal be investigated at the source (PLC logic or UPS feedback wiring) to address the root cause.
On Tue, Apr 28, 2026 at 9:10 PM Kevin Seifert <kevin@autosysnet.com<mailto:kevin@autosysnet.com>> wrote:
@Mason Radke<mailto:mason@autosysnet.com>
Please look into this alarm anomaly.
-Kevin Seifert, AutoSys LLC
________________________________
From: Mike Arias <arias@sslocsd.us<mailto:arias@sslocsd.us>>
Sent: Tuesday, April 21, 2026 12:19 PM
To: Kevin Seifert <kevin@autosysnet.com<mailto:kevin@autosysnet.com>>
Cc: Mychal Jones <mychal@sslocsd.us<mailto:mychal@sslocsd.us>>
Subject: FW:
Good Afternoon Kevin,
So this alarm was received by Mychal Jones around 3:45am Monday. The call did not go to the On Call phone and it did not go to my phone, which it should have gone to both before going to Mychal. Both phones were checked and there were no missed calls. Let us know what you find out. Thank you.
Kind regards,
Michael Arias
Michael J. Arias
Operations Supervisor
Grade III Operator
South San Luis Obispo County Sanitation District
1600 Aloha Place, Oceano, Ca.
805-489-6666
[South San Luis Obispo County Sanitation District]
From: Mike Arias <mike_arias86@yahoo.com<mailto:mike_arias86@yahoo.com>>
Sent: Tuesday, April 21, 2026 12:15 PM
To: Mike Arias <arias@sslocsd.us<mailto:arias@sslocsd.us>>
Subject:
[Photo attachment: 20260421_100131.jpg]
Yahoo Mail: Search, Organize, Conquer<https://mail.onelink.me/107872968?pid=nativeplacement&c=US_Acquisition_YMktg_315_SearchOrgConquer_EmailSignature&af_sub1=Acquisition&af_sub2=US_YMktg&af_sub3=&af_sub4=100002039&af_sub5=C01_Email_Static_&af_ios_store_cpp=0c38e4b0-a27e-40f9-a211-f4e2de32ab91&af_android_url=https://play.google.com/store/apps/details?id=com.yahoo.mobile.client.android.mail&listing=search_organize_conquer>
--
-Mason
---