Reconnect of Vcenter which enabled a re synch with Vcloud,

This may be an indicator of Vcloud DB performance issues and I would recommend performing some clean up of both Vcenter and Vcloud DB's.









Befroe running any of these procedures please make a back up of your DB

Vcenter DB Maintenance
Run this on all Vcenter Instances

1. Stop the vCenter Server Service
2. Stop the vCloud Cell Services for all Cells (Service vmware-vcd stop)
3. Run the following against the vCenter DB:

ALTER TABLE VPX_EVENT_ARG DROP CONSTRAINT FK_VPX_EVENT_ARG_REF_EVENT;
ALTER TABLE VPX_EVENT_ARG DROP CONSTRAINT FK_VPX_EVENT_ARG_REF_ENTITY;
ALTER TABLE VPX_ENTITY_LAST_EVENT DROP CONSTRAINT FK_VPX_LAST_EVENT_EVENT;

truncate table VPX_TASK;
truncate table VPX_ENTITY_LAST_EVENT;
truncate table VPX_EVENT;
truncate table VPX_EVENT_ARG;


alter table VPX_EVENT_ARG add constraint FK_VPX_EVENT_ARG_REF_EVENT foreign key(EVENT_ID) references VPX_EVENT (EVENT_ID) on delete cascade;
alter table VPX_EVENT_ARG add constraint FK_VPX_EVENT_ARG_REF_ENTITY foreign key (OBJ_TYPE) references VPX_OBJECT_TYPE (ID);
alter table VPX_ENTITY_LAST_EVENT add constraint FK_VPX_LAST_EVENT_EVENT foreign key(LAST_EVENT_ID) references VPX_EVENT (EVENT_ID) on delete cascade;

SQL Indexes
Run the SQL Reports, I would suggest just fixing the "Organize" suggestions and running a script to trebuild the indexes for Vcenter

Report
http://blog.sqlauthority.com/2010/03/26/sql-server-generate-report-for-index-physical-statistics-ssms/

Vcenter Index Script
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2009918


-------------------------------------------------------------------------------------


Vcloud Director
The Vcloud Director was set to keep activity history for 90 days, this will of caused the audit_event tables to grow and can have a negative impact on the perfomance of Vcloud Director

https://ikb.vmware.com/contactcenter/php/search.do?cmd=displayKC&docType=kc&externalId=2114814&sliceId=2&docTypeID=DT_KB_1_1&dialogID=828144529&stateId=0%200%20828162863

This statement was provided by the above URL

SET IDENTITY_INSERT audit_event ON;
SELECT * INTO #audit_event_temp FROM audit_event WHERE event_time >= 'yyyy-mm-dd';

TRUNCATE TABLE audit_event;

INSERT INTO audit_event ([internal_id],[event_id],[event_type],[event_time],[status],[user_id],[user_name],[org_id],[org_name],[entity_type],[entity_id],[entity_name],[external_object_id],[external_object_href],[service_name],[description],[details],[product_version]) SELECT [internal_id],[event_id],[event_type],[event_time],[status],[user_id],[user_name],[org_id],[org_name],[entity_type],[entity_id],[entity_name],[external_object_id],[external_object_href],[service_name],[description],[details],[product_version] FROM #audit_event_temp;

DROP TABLE #audit_event_temp;

SET IDENTITY_INSERT audit_event OFF;
----------------------------------------------------

Clear The Vcloud Director DB temporary Inventory Tables

1)) Make a backup of you Vcloud DB
2) Stop the Vcloud Service service vmware-vcd stop on all cell's

SQL
delete from dbo.compute_resource_inv;
delete from dbo.custom_field_manager_inv;
delete from dbo.cluster_compute_resource_inv;
delete from dbo.datacenter_inv;
delete from dbo.datacenter_network_inv;
delete from dbo.datastore_inv;
delete from dbo.datastore_profile_inv;
delete from dbo.dv_portgroup_inv;
delete from dbo.dv_switch_inv;
delete from dbo.folder_inv;
delete from dbo.managed_server_inv;
delete from dbo.managed_server_datastore_inv;
delete from dbo.managed_server_network_inv;
delete from dbo.network_inv;
delete from dbo.resource_pool_inv;
delete from dbo.storage_pod_inv;
delete from dbo.storage_profile_inv;
delete from dbo.task_inv;
delete from dbo.vm_inv;
delete from dbo.property_map;

delete from QRTZ_SCHEDULER_STATE;
delete from QRTZ_FIRED_TRIGGERS;
delete from QRTZ_PAUSED_TRIGGER_GRPS;
delete from QRTZ_CALENDARS;
delete from QRTZ_TRIGGER_LISTENERS;
delete from QRTZ_BLOB_TRIGGERS;
delete from QRTZ_CRON_TRIGGERS;
delete from QRTZ_SIMPLE_TRIGGERS;
delete from QRTZ_TRIGGERS;
delete from QRTZ_JOB_LISTENERS;
delete from QRTZ_JOB_DETAILS;



--------------------------------------

Re-Indexing and Organize the Vcloud Indexes.

Run the SQL Wizard and re-organize any indexes recommended by the report

Script for rebuilding Inexes of the Vcloud DB

USE [vcloud]

SELECT
    RowNum = ROW_NUMBER() OVER(ORDER BY t.TABLE_NAME)  
    ,TableName = t.TABLE_SCHEMA + '.' + t.TABLE_NAME  
    ,AlterMe = 'ALTER INDEX ALL ON [' + t.TABLE_SCHEMA + '].[' + t.TABLE_NAME + '] REBUILD WITH (FILLFACTOR = 90);'
INTO #Reindex_Tables  
FROM INFORMATION_SCHEMA.TABLES t  
WHERE TABLE_TYPE = 'BASE TABLE'
 
DECLARE @Iter INT
DECLARE @MaxIndex INT
DECLARE @ExecMe VARCHAR(MAX)  
 
SET @Iter = 1  
SET @MaxIndex =  
(  
    SELECT COUNT(1)  
    FROM #Reindex_Tables  
)  
 
WHILE @Iter < @MaxIndex  
BEGIN
    SET @ExecMe =  
    (  
        SELECT AlterMe  
        FROM #Reindex_Tables  
        WHERE RowNum = @Iter  
    )  
 
    EXEC (@ExecMe)  
    PRINT @ExecMe + ' Executed'
 
    SET @Iter = @Iter + 1  
END


Start the vCD Service for all 1 cell 1st  (Service vmware-vcd start)
On the Cell, tail the cell.log to ensure the cell comes up to 100%

tail -f /opt/vmware/vcloud-director/logs/cell.log
When the UI of vCD comes up, do a 'Reconnect' action against the vCenter from the System Admin - vCenters section.

Then restart the Vcloud service on any other cell's.