XenMobile – Troubleshooting Log Collection Guide

a. iOS

Logs and data that can be collected from iOS device are

1. SecureHub Logs
2. Xcode logs
3. Device Network Traces
4. Crash Dump

Secure Hub Logs :

To collect SecureHub logs,
• Open Secure Hub
• Tap on Menu / ?
• Tap on Help
• Report issue
• Tap “Citrix Secure Hub”
• Tap “Advanced log settings”
• Select the desired log level, for most issues – please choose Level 15 Debug 10
• Select Log Destination to be file.
• Note : If you need Secure Hub logs to be printed in Xcode, choose Console.
• Enter “Type of issue” and “Describe the issue”
• And send

To Collect Xcode logs
In order to collect logs using xCode package, follow these steps:
• Install XCode
• Connect your iPhone to the Mac
• Select Trust this computer on the iPhone pop-up request
• Start xCode and go to the menu bar: Window → Devices. Select your iPhone and
press the up arrow button in the bottom right.
• Reproduce the problem.
• Press the down arrow button and download the logs.

To Collect Device Network Traces
• Connect your iOS device to your Mac via USB.
• Set up an RVI for that device.
• (This creates a virtual network interface on your Mac that represents the iOS
device’s networking stack)
• Run your macOS packet trace program, and point it at the RVI created in the
previous step.
• To set up an RVI, you should run the rvictl tool as shown below.

* Now that you know the name of the RVI, you can point your packet trace tool at it.
For example, here’s how you might run tcpdump to take a packet trace from the RVI
and write it to the file trace.pcap.

Important: The RVI represents the entire networking stack of the iOS device; you cannot
target a specific interface on the device as you would on the Mac (using the -i option
to tcpdump). However, information about the interface is recorded in the packet
metadata. You can use tcpdumpto display (via the -k option) and filter on (via the –
Q option) this metadata. See the tcpdump online manual for details.
https://developer.apple.com/library/content/qa/qa1176/_index.html

To Collect Crash Dump
1. Launch Xcode
2. Open the Xcode Organizer window. (Window menu -> Organizer, or Cmd-Shift-2.)
3. Find your device in the left sidebar, then select “device logs”.
4. Choose the app and crash report and select “Export” at the bottom of the Organizer
window. This will copy the crash reports to your hard drive.
https://www.chromium.org/developers/how-tos/retrieving-crash-reports-on-ios

b. Android
Following logs can be collected from Android devices.
• Secure Hub logs
• ADB logs
• ANR Logs

To collect SecureHub logs,
• Open Secure Hub
• Tap on Menu / ?
• Tap on Help
• Report issue
• Tap “Citrix Secure Hub”
• Tap “Advanced log settings”
• Select the desired log level, for most issues – please choose Level 15 Debug 10
• Select Log Destination to be file.
• Note : If you need Secure Hub logs to be printed in ADB, choose Console.
• Tap back
• Tap Reproduce issue
• Enter “Type of issue” and “Describe the issue”
• And send

To Collect ADB logs
1. Install Android SDK from http://developer.android.com/sdk/index.html. Make sure
platform-tools is included (http://developer.android.com/sdk/installing/addingpackages.
html).
2. Install usb driver for the phone
(http://developer.sonymobile.com/wportal/devworld/search-downloads/driver)
3. Enable USB Debugging on your device. Go to settings, applications, development and
tick “USB Debugging”.
4. Connect usb cable to the phone
5. Go to the Android SDK directory (for example C:\Program Files\Android\androidsdk\
platform-tools)
6. Type adb shell
7. Type logcat (ref: http://developer.android.com/tools/help/logcat.html)

Example command to collect ADB logs
adb logcat -v “threadtime” > <filename>

ANR Logs
1. From Platform-tools folder where ADB is installed
2. adb pull /data/anr/traces.txt

c. Windows
Generating Field Medic Logs:
1. Run the Field Medic app
2. Tap Advanced.
3. Select the categories that you want to include in the report.(Choose Enterprise)
4. Press the Back button to return to the main Field Medic screen.
5. Tap Start Logging
6. After you start logging, Field Medic displays the elapsed capture time under the Stop
Logging option. Close Field Medic by pressing the Back or Start button. Reproduce
the problem that you want to collect information about.

Note
ETW logging for the selected categories remains enabled, even after the device reboots,
until you stop logging.
You can extract the log from the Window device by connecting it to computer with a USB
cable.
Go to Windows Phone -> Download Folder -> Field Medic and copy the log file.

View log files in a Field Medic report
To extract a Field Medic report, connect the device to a computer with a USB cable. Copy
the Field Medic report from your device (either the root of the device or the root of the SD
card) to your PC. Each folder in this directory represents a different report and contains
several ETW log files.
1. On your computer, locate ETWDump in the Windows Driver Kit.
Example: C:\Program Files (x86)\Windows
Kits\10\ToolFunnel\EtwDump\2.0\EtwDump.exe
2. Locate the ETW manifest files in the Windows Driver Kit.
Example: C:\Program Files (x86)\Windows Kits\10\Manifests
3. The mainifest (.mc) files contain formatting information that ETWDump uses to
decode the log (.etl) files.
Open a Command Prompt window, and make sure the path to ETWDump.exe is on
your Path environment variable.

Here’s an example of using EtwDump to decode the FieldMedic-Contacts-Calendar.etl log
file.
etwdump FieldMedic-Contacts-Calendar.etl -import “C:\Program Files (x86)\Windows
Kits\10\Manifests” -o FmCC.csv –of CSV
ETWDump is one of several tools you can use to decode ETW log files.
Here are some other tools you can use to decode ETW log files:
Xperf
https://msdn.microsoft.com/library/windows/hardware/hh162920.aspx
Tracerpt
https://technet.microsoft.com/library/cc732700.aspx

Additional Logging:
Capture event trace logs on Windows 10 Mobile
https://msdn.microsoft.com/en-us/windows/hardware/commercialize/test/mobiletracing/
capture-event-trace-logs-on-windows-phone
Crash Dump Logs windows 10
• Enable your device for development
https://docs.microsoft.com/en-us/windows/uwp/get-started/enable-your-devicefor-
development
• Crash dumps – In phone’s storage Documents\Debug folder.

III. Active Directory
Authentication Related log collection:
1. Windows Authentication
• Capture the Logon Events under Event Viewer -> Windows Logs -> Security on Windows
Active Directory Server which we have Specified on the XenMobile Server.
• For, additional authentication logging Enable Audit Logon Event under Group Policy, refer
the article https://technet.microsoft.com/en-in/library/cc787567(v=ws.10).aspx

2. Kerberos Authentication
• Capture the Kerberos-related events under Event Viewer -> Windows Logs -> System
• Enable Kerberos logging on the Active Directory Server

o How to enable Kerberos logging
1. Start Registry Editor.
Add the following registry value:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos\Parameters
Registry Value: LogLevel
Value Type: REG_DWORD
Value Data: 0x1
2. If the Parameters subkey does not exist, create it.
Note: Remove this registry value when it is no longer needed so that performance is not
degraded on the computer. Also, you can remove this registry value to disable Kerberos
event logging on a specific computer.
3. Quit Registry Editor
4. Restart the Server for the changes to take effect

IV. Certificate Authority
1. Capture IIS logs
a. How to capture IIS log
• Go to %SystemDrive%\inetpub\logs\LogFiles\W3SVC1\
• Select the u_ex111111 file which relates to the time when the issue occurred
b. To enable advanced logging
• Start -> Run -> inetmgr(IIS Manager) -> Default Sites -> Certsrv -> Failed Request
Tracking Rules -> Add
• Select All Content
• Define status Codes like 401, 403, 500
• Select WWWServer
• Click Finish
• Collect logs from %SystemDrive%\inetpub\logs\LogFiles\W3SVC1\

2. Capture CAPI2 logging
a. How to enable CAPI2 logging
• Start Event Viewer on the Certificate Authority Server
• Under Event Viewer -> Application and Service log -> Microsoft -> Windows ->
CAPI2 -> Operational
• Right click on Operational and select Enable log

V. Exchange ActiveSync
a. Microsoft Exchange Server ActiveSync Logging
1. Capture IIS logs
a. How to capture IIS log
• Go to %SystemDrive%\inetpub\logs\LogFiles\W3SVC1\
• Select the u_ex111111 file which relates to the time when the issue occurred
b. To enable advanced logging
• Start -> Run -> inetmgr(IIS Manager) -> Default Sites -> Microsoft-Server-
ActiveSync -> Failed Request Tracking Rules -> Add
• Select All Content
• Define status Codes like 401, 403, 500
• Select WWWServer
• Click Finish
• Collect logs from %SystemDrive%\inetpub\logs\LogFiles\W3SVC1\

2. For APNS related issue enable EWS logging on Exchange 2010 and below
a. How to enable EWS logging
• Start -> Run -> inetmgr(IIS Manager) -> Default Sites -> EWS -> Logging ->
Enable
• Collect logs from %SystemDrive%\inetpub\logs\LogFiles\W3SVC1\

3. For APNS related issue enable EWS logging on Exchange 2013 and above
The new default log location is as follows:
\ExchangeInstallPath\V15\Logging\EWS
Which if the default installation on the system drive looks like this:
C:\Program Files\Microsoft\Exchange Server\V15\Logging\Ews
• Refer the link below
https://blogs.technet.microsoft.com/karywa/2016/07/15/exchange-server-
20132016-ews-logging-configuration/

b. Lotus Domino ActiveSync Logging
1. Capture the logs from the from the location below:
/<domino data>/traveller/logs/
2. Following Logs needs to be reviewed
• Activity
• Error
• Usage
• SyncML

VI. XenMobile Mail Manager
• XMM log file exists under the location %programfiles%\Citrix\XenMobile Mail
Manager\Logs\
o Copy all the logs from the log file location
• The XenMobile Mail Manager (XMM) provides 4 different logging levels (Error, Info, Warn
and Debug).
• This setting can be configured manually by performing the following steps:
o On the XMM Server, open the file named “nlog.config” located at:
C:\Program Files\Citrix\XenMobile Mail Manager
o Scroll down to the section named “<rules>” and change the parameter “minlevel”
according to the logging level required:
<rules >
<logger name=”*” writeTo=”file” minlevel=”Debug” />
</rules>
o Save the file
o Restart the service (Citrix XenMobile Mail Manager)

VII. XenMobile NetScaler Connector
• XNC log file exists under the location %programfiles%\Citrix\XenMobile NetScaler
Connector\
• Capture Trace on Netscaler
https://support.citrix.com/article/CTX128655

VIII. Database
To run SQL Profiler
1. Open MSSQL Management Studio.
2. Click on Tools -> Choose SQL Server Profiler
3. Login again
4. Choose a Template(Like Tuning)
5. Click on Event Selection
6. Make sure to filter based on DatabaseName column(If there are multiple Databases on
this server)
7. Click on Run
8. Reproduce the issue
9. Click on Stop Button
10. Save file as and save the trace.
https://docs.microsoft.com/en-us/sql/tools/sql-server-profiler/create-a-trace-sql-serverprofiler

Number of connections to the Database DB_Service
SELECT DB_NAME(dbid) AS DB_Service,
COUNT(dbid) AS NumberOfConnections,
loginame
FROM sys.sysprocesses
GROUP BY dbid, loginame
ORDER BY DB_NAME(dbid)

To Check for Deadlocks
SELECT
db.name DB_Service,
tl.request_session_id,
wt.blocking_session_id,
OBJECT_NAME(p.OBJECT_ID) BlockedObjectName,
tl.resource_type,
h1.TEXT AS RequestingText,
h2.TEXT AS BlockingTest,
tl.request_mode
FROM sys.dm_tran_locks AS tl
INNER JOIN sys.databases db ON db.database_id = tl.resource_database_id
INNER JOIN sys.dm_os_waiting_tasks AS wt ON tl.lock_owner_address = wt.resource_address
INNER JOIN sys.partitions AS p ON p.hobt_id = tl.resource_associated_entity_id
INNER JOIN sys.dm_exec_connections ec1 ON ec1.session_id = tl.request_session_id
INNER JOIN sys.dm_exec_connections ec2 ON ec2.session_id = wt.blocking_session_id
CROSS APPLY sys.dm_exec_sql_text(ec1.most_recent_sql_handle) AS h1
CROSS APPLY sys.dm_exec_sql_text(ec2.most_recent_sql_handle) AS h2
GO

IX. NetScaler
Collecting Diagnostic Bundle from Netscaler
https://support.citrix.com/article/CTX127900

Collecting Netscaler traces
https://support.citrix.com/article/CTX128655

• Other way to collect Netscaler Bundle is from XMS
• Login to Admin Console
• Click on Wrench Icon
• Create Support Bundles
• Enable “Support Bundle for NetScaler Gateway”
• Add Netscaler Management IP, Username and Password.
• Select Netscaler for which you want the Support Bundle created and click on Create
• This will be added in the XMS Support Bundle.

To Check policy hits
nsconmsg –d current –g pol_hits

To check Authentication
Live logs
Login to NetScaler CLI
> shell
root@nsgw# tail -F /tmp/aaad.debug
Additionally you could use grep to find specific user related details.
Like,
root@nsgw# tail -F /tmp/aaad.debug | grep -i “username”

Earlier logs
root@nsgw# cat /tmp/aaad.debug
Additionally you could use grep to find specific user related details.
Like,
root@nsgw# cat /tmp/aaad.debug | grep -i “username”

To enable the Cyclic trace use the CLI command below
start nstrace -nf 5 -time 120 -size 0
This particular trace will create 5 files each of them for 120 sec.
This means the files will start getting overwritten after 5 trace files

X. Sharefile
Capture the logs from the location –
C:\inetpub\wwwroot\Citrix\StorageCenter\SC\Logs
In the folder you find several LogFiles:
Cfgsrv_% date% .txt – StorageZone configuration logs
Sc_% Date% .txt – ShareFile StorageZone “ShareFile Data” Logs
CIFS_% Date% .txt – ShareFile StorageZone Connectors for network sharing logs
Sharepoint_% Date% .txt – ShareFile StorageZone Connectors for SharePoint Server
Logs
You can enable extended logging for the individual components (ShareFile data,
Connector for network shares, Connector for SharePoint Server). The file
AppSettingsRelease.config has to be edited for this purpose . This file can be found in
different folders (in standard installation: C: \ inetpub \ wwwroot):
ShareFile data – C: \ inetpub \ wwwroot \ Citrix \ StorageCenter
ShareFile StorageZoneConnector for network shares – C: \ inetpub \ wwwroot \ Citrix \
StorageCenter \ SC \ cifs
ShareFile StorageZone Connector for SharePoint Server – C: \ inetpub \ wwwroot \
Citrix \ StorageCenter \ SC \ sp
In the AppSettingsRelease.config file, change the entry:
<Add key = “enable-extended-logging” value = “0 ” /> to <add key = “enable-extendedlogging”
value = “1 ” />
0 is normal logging and 1 extended logging.

Client tools
The necessary log files for the respective client tools can be found under:
Windows Sync Client:
% Temp% \ Share File
Windows Sync Client configured as OnDemand Sync:
% Systemroot% \ Temp \ Share File
% Temp% \ Share File
Outlook PlugIn:
% AppData% \ Roaming \ Share File \ Outlook
Mac OS X Sync Client:
Users /% username% / Library / Logs / Share File / SyncEngine /

XI. XenApp and XenDesktop

Configuration Logging
http://docs.citrix.com/en-us/xenapp-and-xendesktop/7-13/monitor/configurationlogging.
html

Event logs
http://docs.citrix.com/en-us/xenapp-and-xendesktop/7-13/monitor/event-logs.html