Monthly Archives: June 2012

Device or resource busy when trying to delete vmdk on vmfs

When you try and delete an orphaned vmdk file you get the error:

rm: cannot remove `xxxxxxx-flat.vmdk': Device or resource busy

or

Cannot delete file [xxxxxx] xxxx/xxxx-flat.vmdk

This may be because

  • The file is being used and locked open by a running VM
  • The file is an orphan but is still locked by ESX

To check if the file is an orphan (zombie) download RVtools from

http://www.robware.net/

and click on the vHealth tab. All files that are potentially orphans are listed here.

If the vmdk is an orphan then one of the ESX hosts will be locking the file. Most likely you’ll have quite a few ESX hosts and rebooting each one until the file unlocks would be painful. Instead you can make use of vmkfstools:

SSH to each ESX host in turn and run the following command:

vmkfstools -L release /vmfs/volumes/xxxxxxxx/xxxxxxx/xxxxxx-flat.vmdk

There are two potential errors:

Could not open /vmfs/volumes/xxxxxxxx/xxxxxxx/xxxxxx-flat.vmdk
Command release failed
Error: Device or resource busy

or

Command release failed
Error: Inappropriate ioctl for device

The host that gives you this second error is the one that has locked the file.

Migrate off the VMs from the host and place it into maintenance mode and reboot it. Now try and delete the file that was locked. You should find that the file can now be deleted.

Tagged ,

SQL Server Compressed backups fail with SetEndOfFile error

When performing a backup dump using compression you may get an error like this one:

Executing the query “BACKUP DATABASE [MyDatabase] TO¬† DISK = N’\\\\myserver\\myshare\\…” failed with the following error: “The operating system returned the error ’87(failed to retrieve text for this error. Reason: 15105)’ while attempting ‘SetEndOfFile’ on ‘\\\\myserver\\myshare\\\\MyDatabase_backup_2012_06_12_210001_7306331.bak’.
BACKUP DATABASE is terminating abnormally.
10 percent processed.
20 percent processed.
30 percent processed.
40 percent processed.
50 percent processed.
60 percent processed.
70 percent processed.
80 percent processed.
90 percent processed.
Processed 12767904 pages for database ‘MyDatabase’, file ‘MyDatabase_dat’ on file 1.
100 percent processed.
Processed 9009 pages for database ‘MyDatabase’, file ‘MyDatabase_log’ on file 1.”.
Possible failure reasons: Problems with the query, “ResultSet” property not set correctly, parameters not set correctly, or connection not established correctly.

It doesn’t always happen every backup. The problem appears to be related to the way that SQL server creates the backup file. When using compression the SQL server pre-allocates space for the backup file and the data is then written into it. Since the data is compressed the backup takes less space than was allocated and SQL server instructs the filesystem to truncate the file (set the end of the file). On local disk this can happen quickly but over a network to a share it can take longer.

The function used to truncate the file is SetEndOfFile (http://msdn.microsoft.com/en-us/library/windows/desktop/aa365531%28v=vs.85%29.aspx).

When the backup file is written to a SMB share the truncation can take longer than expected, especially if the amount of data to be truncated is quite large (i.e. it was compressed quite alot) and the request can timeout.

The timeout relating to this on SMB shares is in the Workstation properties (LanManWorkStation). You can tweak the timeout by adding the following registry entry:

1. Click Start, click Run, type regedit.exe, and then click OK.
2. In Registry Editor, locate and then right-click the following registry subkey:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters
3. On the Edit menu, point to New, and then click DWORD Value.
4. Type SessTimeout, and then press ENTER.
5. On the Edit menu, click Modify.
6. In the Edit DWORD Value dialog box, click Decimal under Base, type 300 under Value data, and then click OK.

The default SessTimeout setting is 50 seconds. This changes it to 300 seconds (5 minutes).

For more information check out:

http://support.microsoft.com/kb/961657
http://support.microsoft.com/kb/102067
http://support.microsoft.com/KB/97876

Tagged , , ,

DownloadThemAll firefox addon asks for FTP password for each segment

When downloading a file from an FTP site which requires authentication the DownloadThemAll firefox plugin will ask for authentication when it starts each segment, even if you choose to save it to the password manager.

However, you can work around this by entering the password into the options for the job.

1. Kick off the download through DownloadThemAll as normal

2. Open up the DownloadThemAll manager and right click the job and select Download Information

3. Click the Options tab

4. On the right click on the Manage Download Mirrors icon that looks like this:

5. Click on the URL to expand the full FTP url to the file you are downloading

6. Edit the URL to put in the password so the URL looks like this:

ftp://username:password@ftpserver.mydomain.com/path/to/file.ext

where the password is the password for the username you used to logon to the FTP.

7. Click OK and OK again to get back to the DownloadThemAll manager

8. Do this for each file you are downloading

You will find that when DownloadThemAll starts a new segment it no longer asks for a new password.

Tagged , ,

Utility jobs do not run in Backup Exec 2010 but backups work OK

You see one of these symptoms:

  • When you try to pause the server in Devices you get an error “Unable to pause <servername>”
  • Utility jobs (inventory, eject etc) won’t run and stay in state “queued”
  • Backup jobs show the server state as “paused” but backup jobs still run. Devices shows the server as not paused.

This is an issue with hotfix 354913 in Backup Exec 2010 (not R2 or R3).

http://www.symantec.com/business/support/index?page=content&id=TECH147733

To uninstall the patch in Windows Server 2008

1. Open control panel
2. Programs and Features
3. View Installed Updates
4. Select hotfix -> Uninstall

In other versions of windows go to Add/Remove Programs, find the update and uninstall.

You may need to reboot but I found it worked OK without it.

Tagged ,
Follow

Get every new post delivered to your Inbox.