File Transfer is turned off – Skype for Business Server 2015 Upgrade

After upgrading from Lync Server 2013 to SfB Server 2015 we tried copying and pasting images into IM windows and got the message that File Transfer is turned off.

filetransferoff

I logged into the Lync FrontEnd server to run the command Get-CsFileTransferFilterConfiguration and the action was set to block.

https://docs.microsoft.com/en-us/powershell/module/skype/Set-CsFileTransferFilterConfiguration?view=skype-ps

get-csfiletransfer

Next I started up the SfB Control Panel to look at the Global Policy for FileSharing under IM and Presence, and what I see is that nothing is blocking PNG files, or JPG files, so why can’t I send files?  I also tried allowing peer to peer file transfers under Conferencing, but that didn’t help either.

sfb-controlpanel-fileshare

I ran wireshark on my PC when i was pasting the file/image into the typing area, weirdly enough I didn’t see any traffic going to Lync FE or Edge server, which tells me its a policy in the client that I need to update somewhere, since it isn’t even making it out of my computer.  I tested again, but send a regular text message over Skype and saw the packets go to the FE server.

Resolution: I ended up going to the Conferencing tab and modifying the Global and Desktop and Application Sharing policy to Enable peer-to-peer file transfer and then waiting for it to update and had to restart the Skype client on my PC and the users I was testing with before it started working.

skype-conferencing

Liebert NX 30kVA

Our Liebert UPS started throwing alerts and emailing us every 1-6 minutes.

Events listed in chronological order.
Event: Cleared:Load On Battery
Event: Cleared:Load On Bypass
Event: Active:Load On Battery
Event: Active:Load On Bypass
Event: Active:Inverter DC Voltage Low Shutdown
Event: Cleared:Inverter DC Voltage Low Shutdown
Active Events: 2

The audible alarm on the UPS was going off.  I called our UPS contact who we have a support contract with and got them scheduled to come onsite.  Unfortunately this happened on a Friday at 3:30PM and they were located 2 hours away.  I came back in to meet the tech, who disassembled the UPS, and noticed a slight oil residue on the capacitors and some wires above the caps.

Maker:0x4c,Date:2017-12-3,Ver:4,Lens:Kan03,Act:Lar01,E-Y

The capacitors weren’t something they keep in stock, there are DC caps, which are black and the size of an oil filter for a vehicle, and taller skinnier AC caps.  The AC caps were the ones that had the majority of the oil film on the top of the caps.

Our UPS is currently offline, and switched over to bypass and we are running on straight utility power at this point.  Our building has a generator, but still takes 15 seconds to power on and get up to load to cutover, which is 14.9999 seconds to long.

The tech returned on Monday to replace the caps, powered the UPS back on, and the same issue was still present. We talked with the tech and someone at the support company and the next step was to replace some boards in the UPS, of course they try and upsell us a new UPS at the same time.  The UPS is around 10 years old, in general with regular preventative maintenance, we should be able to get 30 years out of the UPS.

Maker:0x4c,Date:2017-12-3,Ver:4,Lens:Kan03,Act:Lar01,E-Y

According to the tech when they replace the board and cutover, it could possible overload the AC circuit and pop the breakers in the UPS, thus shutting down our entire datacenter.  This has happened to 1 of 4 customers they had the same issue with other customers with.  The whole scenario smells suspicious and the more I am working with them, the less confidence I have in their knowledge.

We called another more local company who works on Liebert, and explained the situation, and of course they said they have never heard of replacing that board causing an overload and popping breakers, so we are waiting to hear back to get a quote for a PM to find out whats actually wrong.  We are working with the original company to see if we can rent some rack mounted UPS’s temporarily to move over everything with dual powersupplies, and then we only have to worry about some network equipment that is redundant by having 2 units, but each unit only has 1 power supply.

We ended up bringing in a Liebert tech directly from Vertiv to look at the issue.  He turned the UPS on, and within 2 minutes, knew what the issue was exactly.  There are 5 hall effect sensors which can go bad overtime, and need to be replaced.

https://en.wikipedia.org/wiki/Hall_effect_sensor

Each sensor was about $80 to be replaced.

 

Lync Server 2013 to Skype for Business 2015 Upgrade caveats and gotchas

After you complete the prerequisites, upgrade the FrontEnd pool, publish the new topology and actually run the SfB 2015 installer, the first window that pops up is to check Microsoft for SfB installer updates.  Foolish me clicked yes check for updates which in turn pulls down the latest CU that gets installed during the upgrade process instead of installing the RTM and then installing the CU after.

The correct option is to “Don’t check for updates right now”

Error encountered: Error returned while installing OcsCore.msi(Feature_LocalMgmtStore), code 1603. Error Message: A fatal error occurred during installation. For more details please consult log at C:\Users\username\AppData\Local\Temp\2\Add-OcsCore.msi-Feature_LocalMgmtStore-[2018_04_23][21_57_03].log

To fix this I uninstalled what SfB components were installed which was only two.  I uninstalled Administrative Tools first, and then Core Components
skype3

Next I started the SfB installer again and selected “Don’t check for updates right now”, and was greeted with the original/legacy installer, and not the upgrade window.  I clicked Install or Update Skype for Business and went through the normal install process, installed certificates, and went to start the services.

skype4

During the startup of the services, the FabricHostSvc and RTCSRV services would not start.  I saw events for the Fabric service which were 23041, 53760, 53761.  I couldn’t find much online for this issue, but it ended up being the Windows Firewall service was disabled on the Lync Frontend service.  I enabled the Windows Firewall service, stopped and started all the SfB services, and then restarting the server and everything came back up.  The Windows Fabric service needs the Windows Firewall service enabled, but the firewall can still be off.

http://stuff4858.rssing.com/chan-59114914/all_p1.html

https://support.microsoft.com/en-us/help/4131777/event-id-32170-skype-for-business-server-director-service-not-start

http://www.thecloudtechnologist.com/error-1603-when-installing-skype-for-business-server-2015/

 

Lync Server 2013 Edge Server running Test-CsTopology and version mismatch

When running Test-CsTopology from a Lync Edge server you get the following error from Lync Server Management Shell.  Test-CsTopology failed. Command setup failed: Cannot find information about the local domain.

This error is to be expected since it can’t pull information since the Edge server is not on the domain.

To ensure replication is happening from the Frontend servers to the Edge server you can run the following command from the Lync Frontend server.

The config will replicate out to the Edge server over port 4443 (so make sure that port is open from the front-ends to the Edge)

Get-CsManagementStoreReplicationStatus

lync-csmanagementstorereplication

https://social.technet.microsoft.com/Forums/lync/en-US/3afee64a-e572-4726-a1e7-d89e530dfd46/cannot-find-information-about-the-local-domain?forum=ocsedge

Also ProductVersion is reporting as 5.0.8308.556 however the servers have been updated to the latest July 2017 CU which is 5.0.8308.992.  To view the actual versions of Lync on the Frontend server run the following command in regular powershell.

Get-WmiObject -query ‘select * from win32_product’ | where {$_.name -like “*Lync Server*”} | Select Name, Version | ft –AutoSize

lync-versions-wmi

wuauclt replaced by usoclient

usoclient is the new Windows 10 and Windows Server 2016 Windows Update orchestrator.  This client has replaced the legacy wuauclt command for previous operating systems.

Below are possible switches you can use with usoclient:

  • StartScan -Used To Start Scan
  • StartDownload -Used to Start Download of Patches
  • StartInstall -Used to Install Downloaded Patches
  • RefreshSettings -Refresh Settings if any changes were made
  • StartInteractiveScan -May ask for user input and/or open dialogues to show progress or report errors
  • RestartDevice -Restart device to finish installation of updates
  • ScanInstallWait -Combined Scan Download Install
  • ResumeUpdate -Resume Update Installation On Boot

Enable-CsTopology : Error accessing folder

When running the Enable-CsTopology command on a Lync Frontend server, a ton of errors pop up stating error accessing the folder.

+ Enable-CsTopology
+ ~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:SourceCollection) [Enable-CsTopol
ogy], DeploymentException
+ FullyQualifiedErrorId : InvalidFolder,Microsoft.Rtc.Management.Deploymen
t.ActivateTopologyCmdlet
Enable-CsTopology : Error accessing folder
\\server\lyncshare\1-ApplicationServer-1\AppServerFiles\Rgs.
At line:1 char:1
+ Enable-CsTopology
+ ~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:SourceCollection) [Enable-CsTopol
ogy], DeploymentException
+ FullyQualifiedErrorId : InvalidFolder,Microsoft.Rtc.Management.Deploymen
t.ActivateTopologyCmdlet
Enable-CsTopology : Error accessing folder
\\server\lyncshare\1-ApplicationServer-1\AppServerFiles\PDP.
At line:1 char:1
+ Enable-CsTopology
+ ~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:SourceCollection) [Enable-CsTopol
ogy], DeploymentException
+ FullyQualifiedErrorId : InvalidFolder,Microsoft.Rtc.Management.Deploymen
t.ActivateTopologyCmdlet
Enable-CsTopology : Error accessing folder
\\server\lyncshare\1-WebServices-1\ABFiles.
At line:1 char:1
+ Enable-CsTopology
+ ~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:SourceCollection) [Enable-CsTopol
ogy], DeploymentException
+ FullyQualifiedErrorId : InvalidFolder,Microsoft.Rtc.Management.Deploymen
t.ActivateTopologyCmdlet
Enable-CsTopology : Error accessing folder
\\server\lyncshare\1-WebServices-1\DeviceUpdateStore.
At line:1 char:1

When running the Lync Server 2013 Best Practices Analyzer I get several issues for Error:Invalid share

invalid-lync-share

Clicking the “Tell me more about this issue and how to resolve it.” takes you to a Microsoft link that doesn’t really tell you anything. https://technet.microsoft.com/en-us/library/?l=%7b0%7d&id=821e8916-7b5d-4f64-8fb0-e5cc392ec1bb(v%3dOCS.15)

After some more research, I found that in order to make configuration changes to the topology, you must be part of the RTCUniversalServerAdmins group. As soon as I added myself to this group, logged out and back in when I run a Test-CsTopology it runs without issue and the Lync BPA best practice analyzer has now cleared that error as well.

Lync Server 2013 CU Install Process

Backups

Export-CsConfiguration -FileName D:\LyncBackups\Config.zip
Export-CsLisConfiguration -FileName D:\LyncBackups\E911Config.zip
Export-CsUserData -PoolFqdn greatlandlync.greatland.com -FileName D:\LyncBackups\ExportedUserData.zip
Copy D:\LyncShare to D:\LyncBackups\LyncShare

Process to install CU
5.0.8308.992 – July 2017 – Latest

CU Installer
Save installer to desktop on LYNCFE1 and LYNCEDGE1
Lync Cumulative Update Installer
https://www.microsoft.com/en-us/download/details.aspx?id=36820

Upgrade LYNCFE1 – Front-end server

  1. Get-CsPoolUpgradeReadinessState (ignore the insufficientactivefrontends error)
  2. Stop-CsWindowsServices –Graceful
    1. Check all Lync services are stopped – Get-CsWindowsService
  3. Run the Cumulative Server Update Installer
    1. Click Install Updates
  4. Restart Server
    1. All Lync services should be running after reboot, may take a bit to start
    2. Run CU installer again and make sure everything is green

Back-end database updates

  1. Lync Server 2013 Monitoring/Archiving/Persistent Chat databases
    1. Test-CsDatabase –ConfiguredDatabases –SqlServerFqdn DB2.company.com –Verbose

LYNCEDGE1 steps

  1. Stop-CsWindowsService –Graceful
    1. Get-CsWindowsService
  2. Run the Cumulative Server Update Installer
    1. Click Install Updates
  3. Restart Server – Ensure all Lync services come back online after reboot