You may be a pro at administering a Domino server, but that doesn't mean you understand everything under the hood. Learn the no-fuss, no-frills, simple explanations of the most common Domino concepts and find out exactly how they work, and how to apply this knowledge to help avoid problems and improve performance. You'll learn how reader fields work, how replication and cluster replication works, mail routing including SMTP, the secrets of ACLs, ID file details, Adminp, busytime, cross-certification, recertifying, Domino domains, transaction logging, view indexes, and more. Learning the entire concept behind the "stuff" that's in Domino will make you a better troubleshooter, a faster problem-solver, and an all-around great admin!

1. JMP105 “How Stuff Works” - Domino Style! Jess Stratton | Independent Consultant | Solace Susan Bulloch | Support SWAT Engineer | IBM

3. Technote/whitepaper/wiki article author

5. Blogger, tweeter (@NerdGirlJess)

6. Written technical articles and been a contributing author for textbooks

7. Currently an independent consultant!

9. This presentation is modeled after the brilliant example of HowStuffWorks.com From HowStuffWorks.com: "Our premise is simple: Demystify the world and do it in a simple, clear-cut way that anyone can understand."

10. How “HowStuffWorks.com” works...

11. Agenda How Notes Named Networks work How Replication works How Cross-Certifying works How Connection Documents work How Clustering works How View Indexes work How Mail Routing works How Busytime works How Transaction Logging works How Domino Domains work How Agent Manager works How Program Documents work How SMTP works How AdminP works How Smart Upgrade works How Access Control Lists work How ID Files work How Readers fields work How Recertifying works

13. Routing and Notes Named Networks

14. Users and Notes Named Networks

15. Setting the Notes Named Network

19. While other uses are possible, the primary usage for having Notes Named Networks is for mail routing scenarios.

23. When users find a replica of a database, it will automatically “find” replicas of databases also based on servers in the same NNN as the user's home server.

25. In the Server doc, go to the Ports-->Notes Network Ports tab.

26. Enter the port name. Domino will automatically set the protocol.

29. Keeping it Simple

30. Tips and Gotchas

39. This will help you avoid nasty troubleshooting problems

41. If the information in this view is stale, stuff won't work

44. Basic Mail Routing

45. The Routing Table

46. Routing Mail Efficiently

47. Transfer Queues and mail.box

53. Router determines the maximum number of threads to allocate

59. View indexing is important!

61. Or restart the router task

63. Plus all servers that use Adjacent domain documents to connect to other domains

65. Since Version 8.5.1

71. You can sometimes see your slow connections!

73. What do servers in a domain share?

74. What are the types of Domino domains?

76. Domino server groups

77. Clustering

80. The Certification Log

81. These databases need replicas of each other on each server in the domain!

83. These can be used for “next-hop” instructions, and also restriction/allow rules.

86. Configuring an Internet domain for SMTP

87. Configuring Domino for SMTP

88. What happens when a message is sent

89. What happens when a message is received

91. In its simplest form, mail is sent from one server directly to another over port 25.

96. Domino must have a Mail.box, or multiple Mail.boxes.

98. Enable SMTP to be used outside the local Internet domain (Config doc)

99. Specify a relay host when sending message outside the local domain (Config doc)

100. Set up mail relay restrictions to avoid spammers (Config doc)

104. The Router task looks up the least-cost route to a Domino server that has “SMTP allowed for all messages”.

105. If enabled, the Router task converts the message to MIME format.

108. This happens with EVERY outbound email through Domino!

110. The Router picks up the incoming message and deposits it into Domino's mail.box

111. The Router looks up the incoming address against the $Users view in the Domino Directory.

112. If it finds a match (and there are no restrictions against the address in the Configuration document) it looks up the user's home server in their Person document.

113. The routing tables finds the least-cost path to deposit the message in the mail.box of the user's home server.

114. Finally, the home server's Router task looks up their mail file in their Person document, and if the server has sufficient access in the ACL, deposits the message into the database.

116. The access levels and user types

117. Roles and readers fields

118. Enforcing a consistent access control list

119. The maximum Internet name and password

120. Understanding -Default-

123. The Domino Directory itself is a database, names.nsf

124. Replication success relies on the servers having the proper access

125. Busytime/calendaring and scheduling relies on being able to access user calendars

127. A user can never be granted more rights in a database than what's been given to them in the ACL, no matter how the database is designed or what the Domino Directory says about them.

128. There are switches to fine-tune each access level, also called privileges .

130. You can read documents that you are listed in an “Author”'s field.

135. Can add/edit the database access control list

137. These are so Domino knows what ID file type it should be expecting to go along with a name/Certifier combination.

138. Leaving a user or group as “Unspecified” in the ACL will make it possible for a user to create a local group titled as a person's name to gain higher access.

141. This allows you to cover your bases, and not leave an entry as “Unspecified”.

145. In fact, this is crucial for Readers fields!

149. Even background agents online use server access to run.

150. Why open a can of worms?

152. Setting it to “Reader” will allow anyone who access it to read the contents, rather than putting in an entire organization's group membership in the ACL.

154. If you don't use Anonymous, Default is used.

156. Readers fields vs. other security measures

157. How do you implement Readers fields?

158. What are some common gotcha's with Readers fields?

160. Date/Time

161. Number

166. We will cover this in the “gotcha's” section!

167. It will only taketh away; you cannot grant a “No Access” user access by listing them in a Readers field.

169. Explicitly listed user names in Canonical format

170. Groups

171. Other field names on the same form containing resolvable names.

174. LocalDomainServers

177. All servers need to be listed as allowable readers too, otherwise the documents are invisible to them, too.

181. Preparing for a replication infrastructure

182. Implementing replication

183. Ways to initiate replication

184. The science behind replication

185. Deletion stubs and replication

188. In the database replication settings, you can specify a subset of documents to replicate, say, for a particular physical work location.

190. Remember: servers need to replicate, even if users don't!

192. Adminp.nsf (The Administration Process)

193. Certlog.nsf (The Certification Log)

194. ... more.

195. These databases need to stay in sync!

197. If any form in the database that will replicate has Readers fields, make sure the servers are listed in those fields.

199. When they will replicate, and how often the replication interval will be

200. Which server initiates the replication

202. Table of server access levels for replication: (from Notes Help) Access level Allows a server to push these changes Assign to Manager ACL settings Database encryption settings Replication settings All elements allowed by lower access levels Servers you want to use as a source for ACL changes. For tight database security, give this access to as few servers as possible. In a hub-and-spoke server configuration, you typically give the hub server Manager access. Designer Design elements All elements allowed by lower access levels Servers you want to use as the source for design changes. Use Manager access instead if you want one server to control ACL and design changes. Editor All new documents All changes to documents Servers that users use only to add and modify documents. In a hub-and-spoke configuration, you typically give the spoke servers Editor access. Author New documents No servers. You don't typically use this access for servers. Reader No changes; server can only pull changes Servers that should never make changes. Servers in the OtherDomainServers group are often given Reader access. Depositor New documents. Also prevents the server from pulling changes. No servers. You don't typically use this access for servers. No Access No changes. Also prevents the server from pulling changes. Servers to which you want to deny access. Servers in the OtherDomainServers group are sometimes given No Access.

206. Highlight all databases to be copied

207. Select Database-->Create Replica(s) from the right Tools pane.

209. “Replicate Server1/Acme names.nsf”

210. “Push Server1/Acme names.nsf “

212. From Program documents

214. Then, each server makes sure they can authenticate with each other through matching, signed certificates.

219. If the OIDs are the same, no changes have been made to either document.

221. The revision history tells the Replicator task whether a conflict exists.

227. Cluster Manager

228. Other Clustering Tasks

229. The Cluster Replicator Task

230. Tweaking CLREPL

231. How all this Cluster Stuff Works

233. Balances the workload between servers

235. The Cluster Database Directory

236. The Cluster Database Directory Manager

237. The Cluster Administrator

238. The Cluster Replicator

239. AdminP

241. Then looks at the cluster membership list for the names of the other servers

245. The names of the servers in the cluster

246. The server availability index of each cluster server, sorted by the most available first

247. The state of the server if it is BUSY, MAXUSERS, or RESTRICTED

253. Adds/removes databases and clustermates

257. CLREPL_POLL_INTERVAL=10

261. How it works

262. Where it looks

263. Clustering Busytime

264. Doing a Free Time Search

265. Tips and Gotchas

267. Schedule Manager (Sched) and Rooms and Resources Manager (RnRMgr)

271. Every server in the cluster contains a replica of this database

273. Its taken from the CLREPLID field in the server documents

284. MailServer Field = server the busytime task is running on

285. MailFile = correct mail file for the user

290. Spaces are hard to see

293. There should be no rep/save conflicts

295. Determines whether to look in its own BUSYTIME.NSF or ask CalConn to go fetch from another server

303. Queues and Rights

304. When it Works

305. Keep an Eye on Stuff

306. Tips and Gotchas

311. For event-triggered agents, the Agent Manager monitors events and then determines when the agents should be invoked

312. It also watches the clock and invokes scheduled agents when the right time comes.

314. Another queue for agents that are scheduled to run (S)

317. Valid values: one minute to 1440 minutes

320. Server does some (run on server)

321. The HTTP task does those invoked by a browser

323. There's an easy way to watch for these

324. Use Domino Domain Monitoring

326. Run more than one instance if queues back up

328. Use Tell Commands for quick results

330. S S 05:04 PM Today agent2 CENTRAL.NSF

332. 1 - Log agent execution events (partially and completely successful)

333. 2 - Log agent execution events (completely successful only)

337. How AdminP requests are created

338. How AdminP requests are processed

339. How AdminP Works

340. Example AdminP Request

341. Tips and Gotchas - AdminP

345. Shares a replica ID with every server in your domain

347. All servers in the domain

348. On specific target servers

350. A user delegates her mail file

351. A server gets upgraded

354. Then the next thing happens

356. Looks for new requests first

357. Then for requests to re-run a request

359. Its called an Administration Process Log document

364. This is adjustable in the Server Document

365. May be needed in times of high activity

367. This isn't even a complicated request

368. See Admin Help for other flow charts

372. What's in an ID file?

373. Who needs an ID file?

374. How public and private key pairs work

375. How encryption keys work

376. How validation and authentication work

378. The ACL gets you access to your databases (using your ID file).

381. Encryption keys

382. Recovery info (if you use it)

383. Public and private key pairs

384. Finally, the user's name and Lotus Notes license number

386. A user who needs to encrypt their mail, or a document.

387. A user who needs to sign an email.

388. A user who needs an encryption key to see the contents of a certain field.

389. A Domino server

394. The key is mailed to the users who need access to the field.

398. Trust any public key from a valid certificate issued by ancestors.

400. If an ID file's public key is trusted, Domino sends a number challenge out.

401. The workstation encrypts the number with the private key from the ID file and sends back a response.

402. The same public key is used to decrypt the number on the server side.

403. If the numbers match, the user's ID file has successfully authenticated.

405. Where is the certificate stored?

406. When would you need to recertify a user?

407. How do you recertify a user?

408. Recertifying vs renaming vs moving certifiers

410. The certificate expiration date

411. The certifier name

412. The public key

413. A digital signature to prove authenticity

414. However, unlike a user or server ID file, a certificate does not contain a private key.

417. It's the contents that's on the physical certifier ID file that must be protected!

420. It will tell you when they will expire.

422. Call it “Certification Log” using the certlog.ntf template.

430. Why use Cross Certification

431. Details of Cross Certification

432. Tips and Gotchas

440. The Public Key

443. When placed into edit mode, the entire public key is visible

446. View indexes vs full-text indexes

447. Updall vs Update

448. How view design affects the Indexer

451. Full-Text services NOS = Notes Object Services. They are portable C++ functions that can access information in databases.

454. A right pointer

456. A “null” pointer is a b-tree element with no elements.

458. This structure allows for incredibly fast lookups and/or insertions.

463. An index sorted by note number.

472. From a program document

474. Discards unused view indexes (every 45 days unless otherwise specified)

479. Benefits of Transaction Logging

480. Requirements for Transaction Logging

481. Under the Hood

482. Types of Transaction Logging

483. Implementing Transaction Logging

484. How it works

491. If using Archive style transaction logging, a dedicated backup and restore system is required

493. Use the fastest, most reliable disks available.

494. Configure the device with a Hot Spare available in case a disk physically fails

498. The Logger records information passed to it by the Recovery Manager

500. Writes transaction-undo to the logger

501. Writes database recovery records to the logger

503. Undoes partially-completed database transactions using transaction-undo from the log

505. DbOpen AutoFixup of Dirty Logged DB F:otusominoataail2addb.nsf

506. Clearing DBIID 0DB5EC6C for DB F:otusominoataail2addb.nsf

507. Completed consistency check on mail2addb.nsf

508. Recovery Manager: Assigning new DBIID for F:otusominoataail2addb.nsf (need new backup for media recovery).

515. After the log fills, the server starts overwriting old transactions

516. Use circular logging if the size of the log needed between full database backup intervals is less than 4GB.

517. Does not support incremental backups

519. Use linear logging if the size of the log needed between full database backup intervals is greater than 4GB

520. And you are not using archive media

522. Simplifies backup and restore

523. Provides online and partial backups

524. The log files are not overwritten until you archive them

527. It fills up the disk space, and then panics

528. Plan accordingly

536. The server stops unexpectedly

538. All is well

540. What can program documents be used for?

541. How to pass parameters to program documents

543. You can specify which Domino server to run it on.

546. Late-night fixups, updalls and compacts

547. Running Domino batch files

548. Replication

549. Scheduled Domino console commands

550. Any command on an individual database

551. When program documents are used, Domino issues a NEW INSTANCE of the program or task.

554. -c “dbcache flush”

556. Command line: - B

557. Program name: nserver

560. The steps to set up Smart Upgrade

561. The trigger and user experience

563. It can upgrade the mail file template at the same time

564. Users can defer the upgrade until a date you specify

568. Point it to a network share, or attach the upgrade files to the kit document.

569. Specify who can use it in the “allowable users” field.

571. 6. (Optional) Create a Desktop policy to upgrade their mail file at the same time.

574. However, as Smart Upgrade tracking is pushed down via policy, the tracking uses DCC

579. If a user is downloading the kit instead of installing across a network share, they will not receive notification to upgrade until the kit has completed downloading and the installation is ready to execute!

581. Please fill out your evals – they DO matter

583. And Wednesday 3PM in Swan 7-10

584. [email_address]

585. She blogs on momelettes.com

587. She's notesgoddess.net

588. Also susan_bulloch@us.ibm.com

590. ID104: IBM Lotus Domino Server Availability: Best Practices and Tuning Tips

591. ID105: DAOS Deployment and Best Practices

592. ID107: Smarter IBM Lotus Domino Monitoring: From Activity Trends to Statistics and Reporting

593. BP101: Adminblast 2011

594. BP103: Got Problems? Let's Do a Health Check

595. BP107: Performing Your Own IBM Lotus Domino Security Review

596. BP108: Admin for the Developer: Build and Secure Your Own IBM Lotus Domino Server Playground -- in an Hour!

597. BP110: Discovering the Mysterious and Dangerous Secrets of STATREP.NSF

598. BP112: 10 Tips to Make You an Admin Star (While Reducing Your Workload)

599. BP116: Backup 101: The What, How, and When

601. BP118: Proactive Server Management: Learn How to Maximize Your Server Uptime

602. BP119: Ground Control to IBM Lotus Notes: Client Management Explained

603. SHOW102: Statistics and Events Base Camp: Proactive Monitoring of Your IBM Lotus Domino Servers

604. SHOW104: Crispy Certificates with Spicy SSL Salsa

605. SHOW109: How To Build a Better Cluster From a Standing Start

606. GEEK101: Speedgeeking!

607. GURU101: GURUpalooza!

608. NERD101: Nerd Girl Panel: Work is Not a Romance Novel or a Football Game

609. Legal Disclaimer © IBM Corporation 2011. All Rights Reserved. The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both. HowStuffWorks™, A Discovery Company is the trademark of HowStuffWorks, Inc. and/or Discovery Communications, LLC, A Silver Spring, MD company in the United States Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. All references to Blossom refer to a fictitious company and are used for illustration purposes only.