Clients/Laguna/evernote/Scada_work_log.md

# Scada work log 220810 Win911 wouldn't call out. Win911 history showed it sent the dial command to the modem but that is all. I went to the device manager and did a query on the modem through the modem driver screen. After that, the call worked. 220708 Still data loss in historian. Found scada PC had network set to public again. 220630 Historian issue Stopped logging data on 6/15 Found these errors in the Log ![[Scada_work_log_image_2.png]] ![[Scada_work_log_image_1.png]] ![[Scada_work_log_image.png]] Stopping historian manually through management console. During stop, messages popped up showing new history blocks were created for the days since the 15th that hadn't been created. Looking further into the log, the IDAS disconnected and reconnected several times, then tried to reconnect the last time but never did. ![[Scada_work_log_image_3.png]] Jerry says SCADA may have been rebooted. Having trouble getting historian to shutdown Shutting down Historian2018 220609 Moved Scada2 and SCADASERVER2018  to the new server room. 220524 Win911 would not start. It would popup a windows error code. Jesse purchased a new year of support for Win911 I copied the MDB file (Configurator>Save AS) and the WIN911.ini file from C:\Program Files(x86)\Specter Instruments\Win-911 V7 Uninstalled Win911 Installed Win911 copied over saved ini file. MDB file was retained. after a reboot, all was working 220324 Changed LIC1000 PID update to 0.25 from 0.01 220318 Changed scaling of Grit chamber level LIT-1000 to 11 ft from 16 ft 220301 Z1 recirc pump P34 is faulting for no apparent reason. 220215 All excel report files with Historian functions are showing #NAME? It must be something wrong with the historian add-in to excel because even the old report files show the same thing. [#NAME? Historian Excel Add-in problem](evernote:///view/17556707/s161/d7c7e1aa-6783-2d58-dbf1-6fd59a1332cc/ea06555b-a98f-406b-8e1a-5d1447301c44/) Excel had disabled the Add-In Added a new login for support using Jerry's email address due to security issue not allowing me to use any other email address other than with the county's domain. Performed updates on Host and took a new snapshot of Historian2018 Copied 2022 report files to Dropbox as a backup Moved Laguna(1) from Dropbox to Box 211012 Fixed Aveva support login, however still cannot access tech notes. Sent them a message 210727 Z4 maintenance clean investigation The train sometimes goes into MCLEAN again after MCLEAN ends. LAD 142 Watched it cycle through MCLEAN and found nothing, it worked fine. If the train is in STANDBY from Low Level Standby or from Low Flow Demand standby, the train will go into MCLEAN. We could try to temporarily turn off this logic to see if it stops. Z4 DECON investigation Train went into STANDBY during refill at around 30 inches tank level. Feed valve was open at 25%. I noticed on the trend that the mode went to 31. The end of DECON sets the mode to 33. So something else caused it to go into STANDBY other than the train tank level. It maybe the SE Pump station low level alarm causing these issues. Robbie and I changed the low level alarm setpoint in ZW PLC from 109 to 105. 210513 Outflow meter FIT-7001 totalizer on SCADA skipped from 75211 to 20211 I did not find a reason why. Added the three tags related to the Outflow meter totalizer to Historian. Historian2018 was having some issues, likely due to low memory assigned. ScadaServer2018 had high memory usage again from SQL. Rebooted ScadaServer2018 Found a couple errors in the Historian log. Tried to log into AVEVA tech support website but something is wrong with the Laguna account. I sent them a message. Backup for Historian2018 VM had a fault. Made room on backup drive through the Altaro manual backup deletion and optimization 210219 RWPS pumps do not shutdown sometimes in automatic mode. Pumps shutdown on low flow setpoint, not on tank level. The flow rate was fluctuating just enough to keep resetting the timer Decreased the Deadband of the PID loop from 10 to 0 Increased the adder (lad17 rung 5) to 50 from 20 The adder acts like a deadband for the shutdown timer reset. 201208 Got the alarm logger to log to the database. Have to set it to open as a normal program, not a windows service. Added the alarm database object to the alarm screen. USE SQL SERVER MANAGEMENT STUDIO TO QUERY DATABASE BEYOND THE 1000 MAX ENTRIES SHOWN  Alarm Database Query: USE [WWALMDB] GO SELECT *   FROM v_AlarmEventHistory GO Moved database to Historian2018  Setup Altaro Backup Upgraded WIMS VM to version 8 201207 Setup Alarm Printer to print alarms to a file in /public/documents Setup a bat file in Startup folder to start the printer on boot Created alarm database on SCADA1B and setup Alarm DB Logger Manager to log all alarms to the DB 201002 Called out due to screens blanking and odd data displayed in SCADA. Similar issue found as 200609-1 note below in the Archestra log. Rebooted RW pump PLC after diagnostics of Linx AB_ETHIP driver showed "commands canceled". The reboot did not fix it. Tried unplugging the ethernet cable from Jesse's computer and that apparently fixed it.  200817 Continuing to troubleshoot RWPS tank level control issue. I'm thinking the PLC must be loosing the tank level signal for some reason. Then I remembered changing the WW DDE topic Advise all items to Advise only active items.  I changed it back to advise all items. I added a level feedback register in the PLC and a new Tag in WW for it. I then added it in place of the FIT-7000 200813 Injection Well report was recording negative daily totals. I looked at the program out there and it seems like everything is working. There have been no negatives in several days. RWPS tank level controls sometimes won't restart the pump after it shuts down. I made a couple PLC changes to help the pumps run longer at a time. 1) changed low flow delay to 1200 sec from 600 2) changed 'scaled min'  PID_RES LN 0007 to the min flow setpoint rather than hard set at 500. 3) changed the LES to LEQ in TANK_LVL LN 0005. 4) Added a ADD block to add 20 to the low flow shutdown setpoint and inserted it into that LEQ to bump up the setpoint to allow for shutdown. 5) lowered min flow setpoint to 500 from 700 with Jeremy's blessing. 200729 Remotely modified UV turbidity and UV_T alarm delays to 120sec per Jeremy Verified that the valve opens on hi-hi turb alarm 200724 Outfall valve has issues. They were PLC logic issues due to 8101 valve. I modified the PLC program to fix it. Other note [Outfall line valves 8101 8102 ](evernote:///view/17556707/s161/16147460-1d11-4471-ba02-04ac287455c3/16147460-1d11-4471-ba02-04ac287455c3/) 200723 RWPS SCADA HOA controls weren't working correctly. Modified PLC and SCADA to change the way the HOA controls are set. Tested RWPS auto functions. 200626 LOWER POND FLOW TOTAL ISSUE (REFERENCE ISSUE FROM 200128) IT APPEARS THAT THE NUMBER REPORTED FROM THE ELPRO UNIT TURNED POSITIVE (ROLLED OVER?). I HAD ADDED LOGIC TO ADD 65535 TO THE TOTAL NUMBER IF IT IS NEGATIVE. THERE WAS NO LOGIC TO TELL IT WHAT TO DO IF THE NUMBER IS POSITIVE. I ADDED LOGIC TO COPY N24:7 TO F8:90 IF IT IS POSITIVE. 200625 DISABLING THESE ALARMS IN WIN911 BY CHANGING "DIGITAL ALARM" TO "NONE" IN "DIGITAL DEFINITION" "ALARM CONDITIONS" AAHH_7010 AAHH_9305 AALL_7020 200623 RWPS NEW VFD AND PUMPS START UP HAD AN ISSUE WITH RSLINX AFTER REPOWERING THE PLC. STATUS ERR \$5 SHOWED ON ALL DATA POINTS IN RSLINX UNDER ACTIVE DDE TOPICS. ENDED UP DELETING THE DDE TOPIC AFTER CLONING IT, THEN RENAMED THE CLONE TO JUST RWPUMPS. THEN I SET THE TOPIC CONFIG IN WW TO ADVISE ALL POINTS. 200615 NEW UV SYSTEM ADDED AN ANALOG OUTPUT TO THE CHEM RM PLC AND WIRED TO THE SIEMENS UV PLC OUTFALL VALVES CHANGED SETPOINTS FOR TURBIDITY HI AND HI-HI TO FLOATING POINT NUMBERS SO THE SETPOINT CAN BE SET BELOW 1. LAD 3 LINE 214,213,215 CHANGED TAG IO POINTS- AAHH_7010_SP from N13:71 to F8:15 AAH_7010_SP from N13:68 to F8:14 Added new tags for F8:15 and F8:16 Having trouble with writing floats to the PLC  200609-2 PREPARING FOR INTOUCH UPGRADE TRACKED DOWN LICENSING INFO FOR WONDERWARE.  I HAD SETUP THE LICENSE SERVER ON SCADASERVER2018 AND UPDATED HISTORIAN TO V2017 BACK IN 2018 **LICENSES ARE INSTALLED AND ACTIVATED ON SCADASERVER2018 FOR ALL WONDERWARE LICENSES V2017** 200609-1 JESSE CALLED YESTERDAY SAYING THERE WAS AN ISSUE WITH SCADA SCREENS "GREYING OUT". THEY REBOOTED THE PC AND IT WAS FIXED. HE CALLED AGAIN THIS MORNING AND THE PROBLEM IS HAPPENING ABOUT EVERY 4 HOURS. LOOKED AT THE WINDOWS LOGS AND DIDN'T FIND MUCH OTHER THAN A COUPLE SINGLE ERRORS FROM VIEW.EXE AND INTOUCH.EXE. LOOKED AT THE WONDERWARE LOGS (SYSTEM MANAGMENT CONSOLE) FOUND ERRORS LOGGING EVERY 2 SECONDS FROM COMPONENT INTSPT "Failed to poke item N14:24 for access name RWPUMPS" ALSO FOR N14:23 AND N14:25 THESE ARE THE THREE INTEGER FILES THAT ARE WRITTEN TO THE RW PUMP PLC FOR TANK LEVEL AND ALARM SETPOINTS. Removed the following 3 lines from Intouch Application Script RW_TANKS_LVL_INT = RWTANKS_LOWEST_LVL * 10; RW_TANKS_LALL_SP1 = RW_TANK02_STPT_LALL * 10; RW_TANKS_LAHH_SP2 = RW_TANK02_STPT_LAHH * 10; RSLINX DDE/OPC TOPIC CONFIGURATION/ADVANCED COMMUNICATION CHANGED "Number of errors before returning error to client" from 0 to 5 RESTARTED INTOUCH AND THE ERRORS STOPPED LOGGING. 200128 Lower pond outfall meter is totalizing in the negative. See separate note. Looking into Z1 inflow valves 191204 Fixed SE pump sump LT signal was jumping. Disconnected the shield on the instrument side of the 4to20 signal. Replaced batteries in SCADA 1 PC UPS. Downloaded HMI program from Zeenogem. Researched refurbished Panelviews. 191004 Adjust minimum PID output for LIC1000 per Jesse from 60 to 50% 190710 Window maker wouldn't run. Said application is being edited in another program. Go to the application folder and delete any appedit.lok files Worked on generator status and alarms 190424 More comm issues on the wireless network. There was 3 to 4 Mbps of traffic that I could not isolate. Moved the SCADA traffic to a new AP. See Network Work Log for more. 190306 Have been having issues on and off with the alarm calls not going out. Reinstalled Win911 on Scada1. Not fixed. Found phone line is dead. Started the process of installing Win911 on the server. Got it installed but not connected to InTouch. I think the problem is that the versions of InTouch on Scada1 and the server are not the same. The tech note from Win911 said to install the same version on both. 190214 Setup Historian data acquisition server to use SCADA2 as a backup IO server.  Created a OneDrive account and a batch file to copy the Virtual_Disks and Virtual_Machines folders to OneDrive  Created a Task in windows to run the batch file every Thursday at 0345  OneDrive can't handle the file size. Setup my portable drive temporarily.  190207 Historian lost connection to SCADA2 again.  The network was "unidentified" and set to private.  Disabling then enabling the network connection fixed it.  Need to change Historian connection to data from SCADA2 INTOUCH to possibly a direct connection to a IO Server Re-setup the Paragon Drive to VHD backup of SCADA2.  190122 Fixed monthly report Fixed lab computer print function.  Troubleshoot Z4 turbidity meter Setup database automatic backup. Didn't finish. Setup a Amazon S3 bucket and [Sqlbackupandftp ](http://sqlbackupandftp.com/)software. It was having a problem.  190107 Adjusted injection well annular psi scaling to 0-100 in SCADA. It was 0-150. When the transmitter got replaced it was a 0-100. I also changed the chart recorder input range.  181126 Worked on migration of Historian. Finally got it to work.  Modifying report spreadsheets.  181120 Worked on Active Directory GPO Added license files to license server Worked on migration of Historian 181106 Historian was blocked from accessing SCADA2B because the network got switched from private to public. Changed the network to private using a method found online.  Installed Historian Excel pluggin  Requested license entitlement files from tech support  181030 Backed up Historian database through sql manager Installed Schneider license server on SS2018 Installed WW Historian 2017 on Historian2018  Configured Historian Installed updates Installed Office 2010 Requested license file from Wonderware  181023 Shutdown WIMS VM and performed an export  Performed backup of Historian Server Tried to Blocked internet access to SCADA2B. The guys were using it for shopping etc. I unpinned IE from the task bar and start menu.  Imported WIMS VM to new server 181003 Installed video card into SCADA2B and setup dual monitors  Cleaned inside SCADA2B  Setup user account on SCADASERVER for IT to use for share and printer Installed update to HISTORIAN2018  Downloaded Aveva tech note TN432 on moving Historian  Downloaded Historian 2017 admin manual (on SCADA2B)  Setup excel file for 2018 injection well data  180928 Troubleshoot WSUS  Set SCADA2B to update from WSUS (doesn't work)  Setup AD and DNS on Server2018 Installed windows updates on Server2018 and Historian2018  Joined Historian2018 to the Laguna.local domain 180921 Performed Veam backup of Historian VM Installed OS on Historian_2018  Setup WSUS on SCADASERVER2018  As of 180921 ![[img_20180816_163412.908.jpg]]