The 4th tutorial on semantic mediawiki for SMWCon Spring 2011 By Jesse Wang, Ning Hu, and Wil Smith

1. Applied Semantic MediaWiki SMWCon Spring 2011 Tutorial #4 2011.04.28

2. Who Are We Jesse Jiaxin Wang Sr. Software Architect, Vulcan Inc. http://semantic-mediawiki.org/wiki/User:Jesse Ning Hu Sr. Software Developer, TeamMersion LLC Wil Smith Software Engineer, Vulcan Inc.

3. About Vulcan 3 Vulcan Inc. was established in 1986 by investor and philanthropist Paul G. Allen, co-founder of Microsoft, to manage his business and philanthropic efforts. Allen is chairman of Vulcan and his sister, Jody Allen, is president and CEO.

4. Agenda

5. Jumpstart

6. Using a Bundle Semantic Bundle http://www.mediawiki.org/wiki/Semantic_Bundle Features: (Pros and Cons) A set of author-selected extensions A fixed set of extensions, few customization options Simplified download and configuration process Still need manual configuration Usually up-to-date and work well together Community tested and supported

7. Installation Package SMW+ Package http://smwforum.ontoprise.com/smwforum/index.php/Help:SMW%2B Features: A set of (fixed) company selected extensions Simplifies installation and configuration Windows installer and VM images are great And they just became free! You may still be curious about what it does… Professionally documented and tested Usually weeks after major release of latest SMW

8. Real-world Deployment When you want a set of custom extensions, you want Your OWN deployment mechanism!

9. Deployment Practices Your own codebase, with checkpoints SVN or Git to get the base, and then Customize the wiki Script or Manual or Mixed A third-party deployment framework and repository Ontoprise: http://smwforum.ontoprise.com/smwforum/index.php/Help:Deployment_Framework RPI: http://code.google.com/p/smwbp/wiki/setup_wiki Referata: http://smw.referata.com/wiki/Category:Packages

10. Build System You own it! Your own selection of extensions And versions of them Own patches, scripts and templates Vulcan: Internal GIT repositories of completed build Versioning and branches between projects Other examples: RPI Best Practice (Google code) Hudson build (PNNL)

11. Wish List A public wiki installation repository with lots of packages Including necessary data Semantically marked up Well documented (or even reviewed) Easy to find and use Federated wiki package store (Wiki AppStore)

12. Customize

13. What to Customize Skins Extensions Schema and Data Templates and forms Anything else you can code… 

14. Skins How to customize a skin Create your skin file in /skins folder within your wiki directory, like Ontoskin.php Finish this skin files following Ontoskin.php skin files There are two classes in this skin files: Inherit from SkinTemplate, set the CSS and template filter. Inherit from QuickTemplate, set the UI style Create skin folder with your skin name in /skins Copy image and CSS files to your skin folder Use your own skin In LocalSettings.php set $wgDefaultSkin = 'wiking_skin'

15. Seahawk Video Skin

16. Skin Customization Example Vulcan Development Wiki and Seahawk wiki. We have customized these two skins for our projects and the key is the method of QuickTemplate. In this method, you can decide what to show and how it shows on your wiki You can customize wiki site like menu links, page titles, CSS and JavaScript files in the data properties of QuickTemplate For example, show wiki side bar in your skin <?phpforeach ($this->data['sidebar'] as $bar => $cont) { ?> <li> <a href="#"><h5><?php $out = wfMsg( $bar );></h5></a> <ul> <?phpforeach($cont as $key => $val) { ?> <li id="<?php echo $val['id'] ?>" > <a href="<?php echo $val['href'] ?>"><?php echo $val['text'] ?></a> </li> <?php } ?> </ul> </li> <?php } ?>

17. Personalized Skin Skin supports customization via __USERNAME__ User can have personalized styles within a skin http://www.mediawiki.org/wiki/Manual:Skin_configuration#User_CSS Tip: A way to turn off Wikipedia banner ad: http://en.wikipedia.org/wiki/User:Jesseone/vector.css

18. Schema and Data “Just do it!” Easy to start Flat, large ontology, at beginning Many properties only valid in certain cases But it’ll get better! “Designer” Ontology Professionally generated (Protégé) More data (restriction) than necessary Data importer isn’t powerful to fully support all OWL features “The middle road” practical schema design: think, adapt, evolve

19. N-ary Relations Type:Record From 1.5 + supports Semantic Search Still no “Allowed values”, no unit, and no other visual customizations Semantic Internal Object extension Adding an object in the middle Using parameterized (patterned) properties Using a triple store (external)

20. Extensions Where to find extensions http://www.mediawiki.org/wiki/Extension_Matrix By supported version (1.16, 1.17, …) By status (beta, stable, …) By type (API, AJAX, hook, user rights…) By new-ness (sort by most recently created time) By talk page new-ness… By most updated versions…

21. Get the Extensions! Some essential ones! http://semantic-mediawiki.org/wiki/Help:MediaWiki_extensions Semantic MediaWiki extensions are quite a few: http://semantic-mediawiki.org/wiki/Help:SMW_extensions Find what you want Not exactly? Then patch it! Nothing close? Then build it! Or find people to build it 

22. Our Experiences We built several wikis (2008-2011) The most useful extension (SF) often needed patches Auto-completion Fixed set of allowed values All values in a category All values having a specific property All values from a query More parameters support (default values, UID, etc.) New functionality in other extensions or SMW Core Aggregation (average, max, sum, etc.) e.g. > and < comparison

23. Security Extensions Know thyself! – What you want, really. Often customers don’t know what they really want Many choices = nothing is perfect From basic to advanced: MediaWiki built-in security Simple Security (Page Security) HaloACL Features Restrictions

24. A Short Story Proposal Management System (Wiki) Users (including privileged users) submit proposals Reviewers (privileged users) rate and review them Reviewers discuss and make decision and assign What security extension should it employ? HaloACL? SimpleSecurity? MediaWiki built-in groups?

25. How to Select | Test Inclusion / transclusion Special:Export Aton/RSS feed, Watch page Listing and Searching Special:Search RecentChanges, AllPages Diff and Revision URL Action links (raw, render) API (???) Action=query? 3rd party APIs Author backdoor? Caching? Files and Images? Redirects? Other extensions?

26. Visualization: Our Path 2008: SRF used to only have few options 2009: Exhibit seemed cool Ported webit Fixed some bugs and enhanced features Runway, TimePlot, multiple rows in Map view etc. 2010: Open Flash Chart More bars, pies, lines, scatter plot… Interaction Richer information display

27. Comparing SRF Options Google Charts (pie and bar) Google does it! (G gets your data, no offline choice…) Static bar and pie, no color options… Exhibit More choices (timeline, map, facet, runway…) Browser compatibility issues OFC (Open Flash Chart) Many bars, charts, plots, even scattered plots Flash required

29. Sidebar Customization Basic: MediaWiki:Sidebar Enhanced: semantic queries, and tree views Advanced: use variables together with queries Custom: Using an extension: DynamicSidebar Customize your own sidebar at User:<username>/Sidebar http://www.mediawiki.org/wiki/Extension:DynamicSidebar Example: http://wiking.vulcan.com/dev/

30. Current Sprint in Sidebar *Current sprint {{#ask: [[Category:Project sprints]] [[Sprint start date::<{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]] [[Sprint end date::>{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]] |format=template |template=Sidebar query template |link=none |}}

31. My Active Sprint Tasks in Sidebar *My tasks {{#ask: [[Category:Project tasks]] [[Project task owner::<q>[[User alias::{{CURRENTUSER}}]]</q>]] [[Project story::<q> [[Project sprint::<q>[[Category:Project sprints]] [[Sprint start date::<{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]] [[Sprint end date::>{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]</q>]] </q>]] [[Project task status::In progress||Not started]] |format=template |template=Sidebar query template |link=none |}}

32. More Extension Examples Twitter Feed http://www.mediawikiwidgets.org/Twitter_Search (example) Flickr Feed http://www.mediawikiwidgets.org/Flickr Amazon Carousel https://widgets.amazon.com/Amazon-Carousel-Widget/ Google Analytics http://www.mediawiki.org/wiki/Extension:Google_Analytics Optify Analytics http://www.optify.net/

33. Templates and Forms MediaWiki template is essential for its structured data handling Editing, especially Template editing, is not trivial Semantic Forms extension greatly lowers the bar Customizing templates and forms can provide very visually appealing pages

34. Form with a Style http://www.thethirdturn.com/w/index.php?title=Form:Driver&action=edit

35. Forms with autocompletion Advanced Auto-Completion on Customized Query Results Basic Auto-Completion on Category Values

36. Story Template To have a field (form link) for users to quickly create tasks that belong to the story (parameter preset) {{#forminput:Project Task |45 | |Create a new task |Project Task[story]={{PAGENAME}}&Project Task[owner]=__USERNAME__&Project Task[status]=Not started }} http://wiking.vulcan.com/dev/index.php/Prepare_for_SMWCon_Spring_2011

37. Example: Daily Report We want to achieve “burn-down” chart in our sprint overview We need: Daily new tasks (created, but not started yet) Daily in-progress tasks (started by not done yet) Daily finished tasks (completed) Task ownership and completion timeline And similar things for bugs (other analysis)

38. Quiz #1: How do you specify Today, Tomorrow in query?

39. Date Magic and Templates Magic words: CURRENTYEAR/MONTH/DAY Some MediaWiki date-related templates to use DATEDIFF, DATECOMP, … NEXTDAY, NEXTDAYDATE, … http://en.wikipedia.org/wiki/Category:Date-computing_templates http://en.wikipedia.org/wiki/Category:ISO_date_templates

40. Quiz #2: How do you do strictly “less than” or “greater than”? (Before 2010.03)

41. Strict Comparison Date query : how to do “In progress” with a finish date “>” today? Extra markup would reduce usability Semantic MediaWiki did NOT support it until 2010.11 We patched Semantic MediaWiki Core Now (as of SMW 1.5.3 + ) we have < and > implemented Use ≤ and ≥ (on ASCII keyboard 242 and 243) And a configuration to turn it on

42. Quiz #3: What if “Today” changes to “Tomorrow” or later dates in your query template? {{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}

43. Freeze the Time Subclusion is your friend! Subclusion = Substitute + inclusion Today's newly started # tasks: [[Report start task count::{{subst:#ask: [[Category:Project tasks]] [[Project task start date::{{{date|}}}]] | format=count |}}]]

44. More Customization Out there in the Wild Wiki World

45. Integration

46. Steps Know what to build Customers, customers, customers! Get your collection of extensions… Customize your wiki (template, forms) Handle the data (schema, query, API, …)

47. Case 1. Proper Wiki Focus on Skin, Form and Template Steps: Pick a (default) skin Design some forms And templates (queries) behind the forms Build starter pages and Go!

48. Biology Ontology Wiki People need discuss about the details related to construct a biology ontology Classes (categories) Properties Relations Textbook analysis Discussion Forms necessary to keep content tidy

49. Ex: Property Form

50. Forms on Namespace http://www.mediawiki.org/wiki/Extension:Semantic_Forms#Based_on_namespace “Easy-peazy, lemons squeezy” ?

51. Case 2: Simple Workflow App Idea: go from one form to another form Need form links in the template May need user rights management

52. Proposal Review Workflow Example User submits a proposal via a form Fill in basic information about the proposal Set a field in the form to be, say, “New” User doesn’t see further information about “Reviewer” or “Owner”, etc. Semantic notification (email and RSS feed) facilitates communication Reviewer finds the form in a canned query Reviewer changes the status to, say, “Approved” Reviewer field (say “Reviewed by”) is (automatically) set Then reviewer assigns the proposal to a owner, say, “Alice” The owner (“Alice”) now sees it and can start work on it Changes the status to “Active” Adds a start date or maybe an estimated end date too

53. Ways to Handle Workflow Add a link to open another form Force edit “review” using another form (e.g. reviewer form) When saving the new form, the category is changed Now we have a different category  a different default form Change a value to include or exclude to another template May need #if (or #switch) statement in template to change (or include/exclude) template values Use “Page has default form” property (Semantic Forms extension)

54. Case 3: WikingDev Wiki Our project management wiki We use it for *all* things it can do: Proposals, ideas, email messages, features, bugs Agile development: milestones, sprints, stories, tasks Code/Feature association: SVN mapped to tasks/bugs Progress analysis: burn-down charts, work calendar Collaboration, Documentation, Demonstration

55. Examples of Data I/O Wiking Development Wiki An Imported email http://wiking.vulcan.com/dev/index.php/Demo_scenarios An email imported as a Project Bug http://wiking.vulcan.com/dev/index.php/Issue_11886_Insert_a_single_property A task uploaded via Outlook with multiple commits http://wiking.vulcan.com/dev/index.php/WikiTags_Release_Structure

56. Microsoft Office Connector Leverage Microsoft Office applications and technology Bring SMW info to Office applications on-demand API for data I/O: add and modify wiki data within Microsoft Office Utilize semantics to improve relevance Smart actions for semantic properties API API Connections Smart Jesse Wang | SemTech 2010

57. Our Agile Project Management (Scrum) 57 Wiking Jesse Wang | SMWCon Amsterdam 2010

58. Wiki Data I/O Wiking development wiki has two external applications Subversion integration Microsoft Office integration Extensions to support it Semantic Wikitags Semantic Connector

59. Data I/O Extensions MediaWiki API – very basic read/write Page Object Model First Data I/O extension to allow access some wiki structure data (links, title, basic template field) SMWWriter Based on POM, support annotations (semantic properties) Data API extension in SMW+ Semantic WikiTags and Semantic Connector Wiki Object Model A powerful and comprehensive object model (Data API) Read/write at finer object level (sentences, template parameters)

60. Wiki Object Model Previous wiki data API efforts are not good enough to build applications There are strong needs for access the structured data and unstructured data in the wiki as a data store From internal and external applications’ point of view A DOM-like approach is reasonable choice Xpath is a great tool and standard Granularity offers flexibility and power People like to operate on known objects rather than parsing and handling wiki texts APIs provide access to both internal and external apps

61. Wiki Object Model Objects Category Link Property Text Magic word HTML tag Sentence Word* Image 61 Page Section Parameter Parameter value Template Template field Parser function List item Table Table cell * Not yet implemented as of April 2011

63. Demo 1: APIs

64. The API is part of MediaWiki API, with our additions to allow get and set (read/write) the content of a wiki page

65. http://wiking.vulcan.com/dev/index.php/Extension:Wiki_Object_Model/Apis

66. Demo 2: Inline Editor

67. A simple inline editor (AJAX styled) to let user modify a specific portion of the wiki page

68. Based on http://wiking.vulcan.com/dev/index.php/Extension:Wiki_Object_Model/Functions62

69. 4. Final Case Study Stanford University CS 227 Car Wiki Developed for Masters level CS class for assignment in “Knowledge Representation Within a Social Context” Over 40 students created pages populating an ontology, created a personal page documenting their automobile, and compiled site statistics into group pages Extra credit was offered to the entire class as a group upon initial wiki population assignment

70. “Dream Car” Wiki Ontology The ontology was left intentionally incomplete and students were encouraged to update the ontology Car manufacturer pages Car make & model pages Personal “Licensed Car” Page Custom properties added to ontology and personal pages

71. Extensions for Car Wiki Standard SMW extension suite downloaded from SMW+ code repository Custom Semantic Results Formatter used with Open Flash Charts Widgets extension included for creating custom JavaScript APIs to other social frameworks

72. Forms and Templates Semantic Forms for creating company, car, and personal vehicle Templates for presenting your car and linking the make & model to Widget templates for social links

73. Demo of Car Wiki Ontology Sample Car Manufacturer : Audi Sample Car Make & Model : Audi A6/A7 Sample Licensed Cars : William_A6, William_A7 Final Chart of Completed Assignment

74. A Wiki in a Week Ontology, Code Installation and Sample Page Population Completed within a Day SMW+ with core extensions Minimal Updating and Bug Fixing of Components Taken from Other Wiki Projects Custom widgets and result formatters Final Population of Wiki Articles Completed Over the Course of a Week Long Assignment

75. Summary Jumpstart Deployment of your bundle/suite/package Code, pseudo-code/meta-data, and data Customize Skins, Styles Extension choices Integrate Know the requirements and data Put everything together and evolve

76. Thank you Questions and/or comments?