I would classify these notifications broadly into three categories.

1. Workflow Notifications: The email notification received when a WorkItem is assigned in the user’s documentum inbox
2. Document Notification: A Read/Write notification for a document for which user has registered
3. Jobs Alerts: This includes
• Job/Agent Exec failure alerts
• Alerts regarding Environment’s health from default jobs

The important point to note is that I am not claiming that these are the only email notifications send by Documentum. Further the categories of email notifications are not the actual scope of this post.

So moving ahead; if we wish to take benefit of these email notifications the most important thing is that our documentum environment should be capable of sending email notifications. This is achieved through a SMTP server. So, the first thing one would like to check is whether a SMTP IP address is defined in the Server Config Object.

SELECT object_name, smtp_server FROM dm_server_config;

If one is lucky enough to have a SMTP Server configured, the next interesting thing would be to verify whether the configured SMTP server is capable of doing what it is supposed to do. In addition it should also be reachable from the Content Server Host.

The following instructions can be used to test it. The text in blue is the user input.

Use Putty to connect to the Content Server Host and login using the Installation Owner’s account. Connect to the SMTP Server using Telnet.

mypc:~> telnet <smtp ip> 25
Trying <smtp ip>...
Connected to <smtp ip>.
Escape character is '^]'.
220 <smtp host> ESMTP Thu, 7 Jul 2011 11:53:25 -0400
HELO Me
250 <smtp host> Hello [<content server host ip>], pleased to meet you
MAIL FROM: from@email.com
250 2.1.0 from@email.com... Sender ok
RCPT TO: my@email.com
250 2.1.5 my@email.com... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
This is a sample test email
.
250 2.0.0 p67FrP9G027974 Message accepted for delivery


You should feel quiet happy if you receive the intended email. Your SMTP server seems to be working just fine.

If the SMTP server is working absolutely fine then we may like to verify whether documentum is able to use the SMTP Server as per expectation. Documentum uses dm_mailwrapper.sh to call the mail program. dm_mailwrapper.sh resides in $DM_HOME/bin and by looking at it’s content it can be figured out that it is called in the following way.

mypc:~>/documentum/product/6.5/bin> ./dm_mailwrapper.sh "This is the Subject" "my@email.com" testcontent

testcontent is a text file which contains the content of the email message.

dm_mailwrapper.sh uses the mail program to send the emails. mail program can be tested using the syntax given in dm_mailwrapper.sh.

mypc:~>/documentum/product/6.5/bin> /bin/mail -s "This is the Subject" "my@email.com" < testcontent

By default all the Documentum email notification are in plain text. Documentum uses dm_event_sender method (dm_method) which by default uses dm_event_sender.ebs to generate the email content. The Verb of dm_event_sender method can be modified to use dm_html_sender.ebs and it starts generating the email content in HTML. But generating HTML content is not enough. sendmail program can be used in order to make sure that the email is rendered as HTML. dm_mailwrapper.sh can be updated as follows.

#/bin/mail -s "$subject" "$address" < $content_file
/usr/lib/sendmail -io "Dummy Subject" "$address" < $content_file

Before updating the dm_mailwrapper.sh do make sure that the sendmail program is sending emails. The content of testcontent can also be updated as HTML while doing so.

With the above two changes in place we should start getting the email notifications in HTML format. No, there is no need to restart anything. I indeed got few issues with the content of Workflow and the Jobs notification. Those issues were fixed by commenting few lines in the dm_html_sender.ebs. The content of the email notification can be customized by customizing dm_event_sender.ebs for plain text notifications and dm_html_sender.ebs for HTML notifications.

Prerequisites:

• As it’s a HA environment, the content files should be present in a File Store that is shared across the Content Servers.
• The Installation Owner and the Installation Path should be same on each Content Server.
• Availability of a Database Server and its connectivity through each Content Server Host using Oracle Client.
• Update the /etc/hosts file of the Content Server Hosts so that they can resolve their IP addresses and hostnames.

Once the above prerequisites are satisfied, the below steps can be used to establish the HA environment.

• Install the Primary Content Server as per the standard procedure mentioned in Installation Guide.
• Install the docbroker on the Secondary CS host.
• Create a Secondary Server Config object using Documentum Administrator.
• Copy server.ini, aek.key, dbpasswd.txt, dm_start_docbase and dm_shutdown_docbase from Primary CS Host to Secondary CS Host.
• Update the server.ini on both the Hosts so that the docbrokers project to each other.
  server.ini on the Primary CS:

  [DOCBROKER_PROJECTION_TARGET]
  host = <primary docbroker>
  port = 1489
  [DOCBROKER_PROJECTION_TARGET_1]
  host = <secondary docbroker>
  port = 1489

  server.ini on the Secondary CS:

  [DOCBROKER_PROJECTION_TARGET]
  host = <secondary docbroker>
  port = 1489
  [DOCBROKER_PROJECTION_TARGET_1]
  host = <primary docbroker>
  port = 1489

• Update the dm_shutdown_docbase as follows: 
  The line preceding to “shutdown,c,T,T” should be updated as follows:
  • Original:

    ./iapi <docbase> -U$DM_DMADMIN_USER -P -e << EOF 

  • Updated:

    ./iapi <docbase>.<secondary server config object name> -U$DM_DMADMIN_USER  -P -e << EOF

• Update the dfc.properties of the Web Application as well as both the Content Server Hosts so that they point to both the docbrokers.
• Create an ACS Config object using the below command:

  dmbasic -f dm_acs_install.ebs -e Install -- <docbase name> <installation owner> <password> <new acs config name> <secondary server config name> <JMS Port> <JMS protocol> <output log location>

• Update the acs.properties accordingly.
• Once the above steps are complete, shutdown the Primary CS and test the Secondary CS for expected functionality. There may be minor issues which may need few basic fixes. Such fixes include creation of sysadmin directory at $DOCUMENTUM/dba/logs/<repository id>/ in order to fix issues relating to jobs running on Secondary CS.

Start the Primary CS and now both the CS should be in HA.

A1)SELECT COUNT(DISTINCT user_name) FROM dm_user >> 168241

A2)SELECT COUNT(DISTINCT users_names) FROM dm_group >> 2916

A3)SELECT COUNT(DISTINCT user_name) FROM dm_user
WHERE user_name NOT IN
(SELECT DISTINCT users_names from dm_group) >> 0

Isn’t that weird?

B1)SELECT COUNT(DISTINCT user_name) FROM dm_user, dm_group
WHERE ANY dm_group.users_names = dm_user.user_name >> 2915

A2 VS B1: again weird!

C1)SELECT COUNT(DISTINCT user_name) FROM dm_user
WHERE user_name NOT IN
(SELECT DISTINCT user_name FROM dm_user, dm_group
WHERE ANY dm_group.users_names = dm_user.user_name) >> 165326

(A2, A3) VS (B1, C1): any explanation?
But that’s a relief indeed. That’s the result I needed. A1 – B1 = C1. Under the current circumstances, that’s encouraging enough to get into some more meddling.

D1)SELECT COUNT(users_names) FROM dm_group
WHERE ANY users_names NOT IN
(SELECT user_name FROM dm_user) >> 5

What the hell!!!

D2)SELECT users_names FROM dm_group
WHERE ANY users_names NOT IN
(SELECT user_name FROM dm_user)
>> TestUser1 test3 test2 orts test1x

hmmmm…

D3)SELECT COUNT(*) FROM dm_user
WHERE user_name IN
(SELECT users_names FROM dm_group
WHERE ANY users_names NOT IN
(SELECT user_name FROM dm_user))>> 4

This is insane. Isn’t D3 a contradiction in itself? Can I challenge EMC to explain that? Can someone come to my rescue?

D4)SELECT user_name FROM dm_user
WHERE user_name IN
(SELECT users_names FROM dm_group
WHERE ANY users_names NOT IN
(SELECT user_name FROM dm_user))
>> TestUser1 test3 test2 orts

Here is something that helps; but it doesn’t explain the insanity though.

ConsistencyChecker Report:

Summery:
Weirdness No. 1:     (A1, A2, A3)
Weirdness No. 2:     A2 VS B1: Explained by the ConsistencyChecker Report.
Weirdness No. 3:     (A2, A3) VS (B1, C1)
Weirdness No. 4:     D3: The Contradiction in itself.

I could get an explanation only for Weirdness No. 2 which I guess is not a weirdness at all. I hope someone reading this post would try explaining the other three. 1 & 2 I guess are contributed by repeating attributes but remains unexplained anyway. 4, the D3 is an absolute marvel. Is there a bug in the way DQL works?
C1 is the undisputed winner as it provides the expected result.

My first reaction to his query was: “You are expecting too much!”. Life can’t be that easy. You can not escape from every complexity. But we can indeed try to make them simple. Can’t we?
So I looked into dmi_dd_type_info, dmi_dd_attr_info and finally in dm_type I found the attributes that I could use. After a few iterations I came up with the following DQL Query:

SELECT name, attr_name, attr_repeating
FROM dm_type
WHERE name = 'my_document' AND attr_repeating = 1
ENABLE(ROW_BASED)

This query is again a good example of the DQL Hint ROW_BASED. With this Hint we get only those results where attr_repeating = 1. That statement may have sounded dumb had attr_repeating been a single valued attribute. But it is repeating and there are some weirdness with the result if we are checking for the value of a repeating attribute in the WHERE clause without using that hint.

SELECT name, attr_name, attr_repeating
FROM dm_type
WHERE name = 'my_document' AND ANY attr_repeating = 1

The result of the first query brought a smile to Gaurav’s face; but his expectations had increased. He was interested in only those repeating attributes that were not inherited from the parent type. We found a simple way to do that.

SELECT name, attr_name, attr_repeating
FROM dm_type
WHERE name = 'my_document' AND attr_repeating = 1 AND attr_name
NOT IN (SELECT attr_name FROM dm_type WHERE name = 'dm_document')
ENABLE(ROW_BASED)

We came up with one more modification and our final query looked like:

SELECT name, attr_name, attr_repeating
FROM dm_type
WHERE name = 'my_document' AND attr_repeating = 1 AND attr_name
NOT IN (SELECT attr_name
	FROM dm_type
	WHERE name =
		(SELECT super_name
		FROM dm_type
		WHERE name = 'my_document')
	)
ENABLE(ROW_BASED)

Now that has brought a smile to my face and I hope that Gaurav doesn’t come up with more expectations.

*****************************************************************************************************************

SELECT COUNT(*) FROM dm_user
Result  >>  435820

SELECT COUNT(*) FROM dm_group
Result  >>  267375

SELECT COUNT(*) FROM dm_user WHERE r_is_group = true
Result  >>  263559

SELECT COUNT(*) FROM dm_user WHERE r_is_group = false
Result  >>  172261

SELECT COUNT(*) FROM dm_group
WHERE group_name
IN (SELECT user_name FROM dm_user WHERE r_is_group = true)
Result  >>  263559

SELECT COUNT(*) FROM dm_group
WHERE group_name
NOT IN (SELECT user_name FROM dm_user WHERE r_is_group = true)
Result  >>  3816

SELECT COUNT(*) FROM dm_group
WHERE r_object_id
IN (SELECT r_object_id FROM dm_user WHERE r_is_group = true)
Result  >>  0

SELECT COUNT(*) FROM dm_group, dm_user
WHERE dm_group.group_name = dm_user.user_name
Result  >>  465587

SELECT dm_user.user_name,dm_group.group_name
FROM dm_group, dm_user
WHERE dm_group.group_name = dm_user.user_name
Total Results  >>  465587

SELECT DISTINCT dm_user.user_name, dm_group.group_name
FROM dm_group, dm_user
WHERE dm_group.group_name = dm_user.user_name
Total Results  >>  198518

SELECT COUNT(*), dm_group.group_name
FROM dm_group, dm_user
WHERE dm_group.group_name = dm_user.user_name
GROUP BY dm_group.group_name
HAVING count(*) > 1
Total Results  >>  38608

SELECT group_name, COUNT(*) FROM dm_group
GROUP BY group_name HAVING COUNT(*) > 1
Total Results  >>  38608

SELECT user_name, COUNT(*) FROM dm_user
GROUP BY user_name HAVING COUNT(*) > 1
Total Results  >>  39468

*********************************************************************************************************

SELECT COUNT(DISTINCT user_name) FROM dm_user
Result  >>  366103

SELECT COUNT(DISTINCT group_name) FROM dm_group
Result  >>  198518

*********************************************************************************************************

SELECT COUNT(DISTINCT user_name) FROM dm_user WHERE r_is_group = true
Result  >>  195107

SELECT COUNT(DISTINCT user_name) FROM dm_user WHERE r_is_group = false
Result  >>  170996

SELECT COUNT(DISTINCT group_name)
FROM dm_group
WHERE group_name
IN (SELECT user_name FROM dm_user WHERE r_is_group = true)
Result  >>  195107

SELECT COUNT(DISTINCT group_name)
FROM dm_group
WHERE group_name
NOT IN (SELECT user_name FROM dm_user WHERE r_is_group = true)
Result  >>  3411

SELECT DISTINCT dm_user.user_name, dm_group.group_name
FROM dm_group, dm_user
WHERE dm_group.group_name = dm_user.user_name
Total Results  >>  198518

SELECT DISTINCT dm_user.user_name, dm_group.group_name
FROM dm_group, dm_user
WHERE dm_user.r_is_group = true and dm_group.group_name = dm_user.user_name
Total Results  >>  195107

SELECT DISTINCT dm_user.user_name, dm_group.group_name
FROM dm_group, dm_user
WHERE dm_user.r_is_group = false and dm_group.group_name = dm_user.user_name
Total Results  >>  3411

SELECT COUNT(DISTINCT group_name)
FROM dm_group
WHERE group_name
IN (SELECT user_name FROM dm_user WHERE r_is_group = false)
Result  >>  3411

SELECT COUNT(DISTINCT group_name)
FROM dm_group
WHERE group_name
IN (SELECT user_name FROM dm_user)
Result  >>  198518

***************************************************************************************************************

***************************************************************************************************************

It’s just an observation and I don’t have much to comment.