SiSU

git

[ document manifest ]

<< previous toc next >>

SiSU - Manual,
Ralph Amissah

What is SiSU?

Introduction - What is SiSU?

How does sisu work?

Summary of features

Help

SiSU Manual
SiSU man pages
SiSU built-in interactive help, [discontinued]

Commands Summary

Description
Document Processing Command Flags

command line modifiers

database commands

Command Line with Flags - Batch Processing

Introduction to SiSU Markup  3 

Summary
Markup Rules, document structure and metadata requirements
Markup Examples
Online
Installed

Markup of Headers

Sample Header
Available Headers

Markup of Substantive Text

Heading Levels
Font Attributes
Indentation and bullets
Hanging Indents
Footnotes / Endnotes
Links
Naked URLs within text, dealing with urls
Linking Text
Linking Images
Link shortcut for multiple versions of a sisu document in the same directory tree
Grouped Text / blocked text
blocked text curly brace syntax
blocked text tic syntax
Tables
Poem
Group
Code
Additional breaks - linebreaks within objects, column and page-breaks
line-breaks
page breaks
Bibliography / References
a markup tagged metadata bibliography section
Tagging citations for inclusion in the Bibliography
Glossary
Book index

Composite documents markup

Substitutions

SiSU filetypes

.sst .ssm .ssi marked up plain text
sisu text - regular files (.sst)
sisu master files (.ssm)
sisu insert files (.ssi)
sisupod, zipped binary container (sisupod.zip, .ssp)

Configuration

Configuration files
config.yml
sisu_document_make

CSS - Cascading Style Sheets (for html, XHTML and XML)

Organising Content - Directory Structure and Mapping

Document Source Directory
General Directories
Document Output Directory Structures
Output Directory Root
Alternative Output Structures
By Language
By Filetype
By Filename
Remote Directories
Sisupod

Homepages

Home page and other custom built pages in a sub-directory

Markup and Output Examples

Markup examples
SiSU Markup Samples

SiSU Search - Introduction

SQL

populating SQL type databases

Postgresql

Name
Description
Synopsis
Commands
create and destroy database
import and remove documents

Sqlite

Name
Description
Synopsis
Commands
create and destroy database
import and remove documents

CGI Search Form

Setup search form
Search - database frontend sample, utilising database and SiSU features, including object citation numbering (backend currently PostgreSQL)
Search Form

sisu_webrick

Name
Synopsis
Description
Summary of man page
Document processing command flags

Remote Source Documents

Remote Document Output

Remote Output

commands
configuration

Remote Servers

Download information

Download SiSU - Linux/Unix

SiSU Current Version - Linux/Unix
Source (tarball tar.gz)
Git (source control management)
Debian
RPM

Source tree

run off source package directory tree (without installing)
Gem install
Gem install with qi (quick install) script
Gem install with rake
misc Gem
direct installation with qi (quick install) script
installation with setup.rb

Unix/Linux Distribution

Debian

Dependencies

Quickstart - Getting Started Howto

Installation
Debian Installation
RPM Installation
Installation from source
Testing SiSU, generating output
basic text, plaintext, html, XML, ODF, EPUB
LaTeX / pdf
relational database - postgresql, sqlite
Getting Help
The man pages
Built in help
The home page
Markup Samples

HowTo

Getting Help

SiSU "man" pages

Setup, initialisation

initialise output directory
Use of search functionality, an example using sqlite
misc
url for output files -u -U
toggle screen color
verbose mode
quiet mode
maintenance mode intermediate files kept -M
start the webrick server
remote placement of output

Configuration Files

Markup

Headers
Font Face
Bold
Italics
Underscore
Strikethrough
Endnotes
Links
Number Titles
Line operations
Tables
Grouped Text
Composite Document

Change Appearance

Skins
CSS

Extracts from the README

README

Online Information, places to look
Installation
Debian
RPM
Source package .tgz
to use setup.rb
to use install (prapared with "Rake")
to use install (prapared with "Rant")
Dependencies
Quick start
Configuration files
Use General Overview
Help
Directory Structure
Configuration File
Markup
Additional Things
License
SiSU Standard

Extracts from man 8 sisu

Post Installation Setup

Post Installation Setup - Quick start
Document markup directory
Configuration files
Debian INSTALLATION Note
Document Resource Configuration

FAQ - Frequently Asked/Answered Questions

Why are urls produced with the -v (and -u) flag that point to a web server on port 8081 ?
I cannot find my output, where is it?
I do not get any pdf output, why?
Where is the latex (or some other interim) output?
Why isn't SiSU markup XML
LaTeX claims to be a document preparation system for high-quality typesetting. Can the same be said about SiSU?
Can the SiSU markup be used to prepare for a LaTex automatic building of an index to the work?
Can the conversion from SiSU to LaTeX be modified if we have special needs for the LaTeX, or do we need to modify the LaTeX manually?
How do I create GIN or GiST index in Postgresql for use in SiSU
Are there some examples of using Ferret Search with a SiSU repository?
Have you had any reports of building SiSU from tar on Mac OS 10.4?
Where is version 1?
What is the difference between version 1 and 2?

Who might be interested in the SiSU feature set?

Work Needed

Wishlist

Editor Files, Syntax Highlighting

Help Sources

man pages
sisu generated output - links to html
www.sisudoc.org
man2html
locally installed
www.jus.uio.no/sisu

Endnotes

Endnotes

Index

Index

Metadata

SiSU Metadata, document information

SiSU - Manual,
Ralph Amissah

What is SiSU?

Extracts from man 8 sisu

FAQ - Frequently Asked/Answered Questions

Why are urls produced with the -v (and -u) flag that point to a web server on port 8081 ?
I cannot find my output, where is it?
I do not get any pdf output, why?
Where is the latex (or some other interim) output?
Why isn't SiSU markup XML
LaTeX claims to be a document preparation system for high-quality typesetting. Can the same be said about SiSU?
Can the SiSU markup be used to prepare for a LaTex automatic building of an index to the work?
Can the conversion from SiSU to LaTeX be modified if we have special needs for the LaTeX, or do we need to modify the LaTeX manually?
How do I create GIN or GiST index in Postgresql for use in SiSU
Are there some examples of using Ferret Search with a SiSU repository?
Have you had any reports of building SiSU from tar on Mac OS 10.4?
Where is version 1?
What is the difference between version 1 and 2?

Why are urls produced with the -v (and -u) flag that point to a web server on port 8081 ?

Try the following rune:

This should start the ruby webserver. It should be done after having produced some output as it scans the output directory for what to serve.

I cannot find my output, where is it?

The following should provide help on output paths:

I do not get any pdf output, why?

SiSU produces LaTeX and pdflatex is run against that to generate pdf files.

If you use Debian the following will install the required dependencies

the following packages are required: tetex-bin, tetex-extra, latex-ucs

Where is the latex (or some other interim) output?

Try adding -M (for maintenance) to your command flags, e.g.:

this should result in the interim processing output being retained, and information being provided on where to find it.

should also provide some relevant information as to where it is placed.

Why isn't SiSU markup XML

I worked with text and (though I find XML immensely valuable) disliked noise ... better to sidestep the question and say:

SiSU currently "understands" three XML input representations - or more accurately, converts from three forms of XML to native SiSU markup for processing. The three types correspond to SAX (structure described), DOM (structure embedded, whole document must be read before structure is correctly discernable) and node based (a tree) forms of XML document structure representation. Problem is I use them very seldom and check that all is as it should be with them seldom, so I would not be surprised if something breaks there, but as far as I know they are working. I will check and add an XML markup help page before the next release. There already is a bit of information in the man page under the title SiSU VERSION CONVERSION

sisu --to-sax [filename/wildcard]

sisu --to-dom [filename/wildcard]

sisu --to-node [filename/wildcard]

The XML should be well formed... must check, but lacks sensible headers. Suggestions welcome as to what to make of them. [For the present time I am satisfied that I can convert (both ways) between 3 forms of XML representation and SiSU markup].

sisu --from-xml2sst [filename/wildcard]

LaTeX claims to be a document preparation system for high-quality typesetting. Can the same be said about SiSU?

SiSU is not really about type-setting.

LaTeX is the ultimate computer instruction type-setting language for paper based publication.

LaTeX is able to control just about everything that happens on page and pixel, position letters kerning, space variation between characters, words, paragraphs etc. formula.

SiSU is not really about type-setting at all. It is about a lightweight markup instruction that provides enough information for an abstraction of the documents structure and objects, from which different forms of representation of the document can be generated.

SiSU with very little markup instruction is able to produce relatively high quality pdf by virtue of being able to generate usable default LaTeX; it produces "quality" html by generating the html directly; likewise it populates an SQL database in a useful way with the document in object sized chunks and its meta-data. But SiSU works on an abstraction of the document's structure and content and custom builds suitable uniform output. The html for browser viewing and pdf for paper viewing/publishing are rather different things with different needs for layout - as indeed is what is needed to store information in a database in searchable objects.

The pdfs or html produced for example by open office based on open document format and other office/word processor suits usually attempt to have similar looking outputs - your document rendered in html looks much the same, or in pdf. .. sisu is less this way, it seeks to have a starting point with as little information about appearance as possible, and to come up with the best possible appearance for each output that can be derived based on this minimal information.

Where there are large document sets, it provides consistency in appearance in each output format for the documents.

The excuse for going this way is, it is a waste of time to think much about appearance when working on substantive content, it is the substantive content that is relevant, not the way it looks beyond the basic informational tags - and yet you want to be able to take advantage of as many useful different ways of representing documents as are available, and for various types of output to to be/look as good as it can for each medium/format in which it is presented, (with different mediums having different focuses) and SiSU tries to achieve this from minimal markup.

Can the SiSU markup be used to prepare for a LaTex automatic building of an index to the work?

Has not been, is of interest though the question on introducing such possibilities is how to keep them as unobtrusive as possible, and as generically relevant as possible to other output formats (which is why the focus on object numbers). Unobtrusive refers both to the markup (where there is no big problem with introducing optional extras); and, more challengingly how to minimise impact on competing ideas/interests, such allowing the addition of semantic tags which could be tied to objects, mapped against the objects that contain them, (permitting mapping and mining of content in various ways that would be largely agnostic of output format - object numbering being an attempt to move beyond output format based content locators (such as page numbers). The desire being to (be a meta markup and) maintain agnosticism as to what is being generated and in development to favor solutions of that nature. Keep bridging LaTeX, XML, SQL ... make use of objects and serialisation for mapping whether against content or meta-content (such as semantic [or additional structural] markers).

Can the conversion from SiSU to LaTeX be modified if we have special needs for the LaTeX, or do we need to modify the LaTeX manually?

Should be possible to modify code, it is GPLv3, should be possible either to modify existing modules or write an independent module for generating bespoke latex. Generic improvements are welcome for inclusion/incorporation in the existing code base.

If there are tools to generate mathematical/scientific formula from latex to images (jpg, png), the latex parser could conceivably be used to make these available to other output formats.

How do I create GIN or GiST index in Postgresql for use in SiSU

This at present needs to be done "manually" and it is probably necessary to alter the sample search form. The following is a helpful response from one of the contributors of GiN to Postgresql Oleg Bartunov 2006-12-06:

"I have tsearch2 slides which introduces tsearch2 ‹http://www.sai.msu.su/~megera/wiki/tsearch2slides

FTS in PostgreSQL is provided by tsearch2, which should works without any indices (GiST or GIN) ! Indices provide performance, not functionality.

In your example I'd do ( simple way, just for demo):

0. compile, install tsearch2 and load tsearch2 into your database

cd contrib/tsearch2; make&&make&&install&&make installcheck; psql DB < tsearch2.sql

1. Add column fts, which holds tsvector

alter table documents add column fts tsvector;

2. Fill fts column

update document set fts = to_tsvector(clean);

3. create index - just for performance !

create index fts_gin_idx on document using gin(fts);

4. Run vacuum

vacuum analyze document;

That's all.

Now you can search:

select lid, metadata_tid, rank_cd(fts, q,2)as rank from document, plainto_tsquery('markup syntax') q where q @@ fts order by rank desc limit 10;

Are there some examples of using Ferret Search with a SiSU repository?

Heard good things about Ferret, but have not used it. The output directory structure and content produced by SiSU is very uniform. Have looked at a couple of other engines (hyperestraier, lucene). There it was enough to identify the files that needed to be indexed and pass them to the search indexing tool. Some Unix rune doing the job, such as:

find /home/ralph/sisu_www -type f | \
egrep '/sisu_www/(sisu|document_archive)/.+?.html$' | \
egrep -v '(doc|concordance).html$' | \
estcmd gather -sd casket -

you would have to experiment with what gives the desired result, the file doc.html is the complete text in html (there are additional smaller html segments), and plain.txt the document as a text file. It may be possible to index the text file and return the html document.

Have you had any reports of building SiSU from tar on Mac OS 10.4?

None. In the early days of its release a Mac friend built and run the ruby code part that did not rely on system calls to bits like the latex engine. That is already some years back. He was not into writing or document markup, and did it as a favour at the time. I have not followed up that thread of development.

It should however be possible, much of the output relies on plain ruby, and the system commands to latex etc. could be made appropriate for the underlying OS.

Where is version 1?

Version 1 was finally released on December 21, 2009, and largely to make a version 1 branch available as version 2 was in the pipeline with an imminent release. Most of SiSU was mature and stable long before the release of version 1.

What is the difference between version 1 and 2?

Input and output of version 1 and 2 are largely the same with the following significant exceptions. On the input side, document headers, that is metadata and processing instructions have changed in version 2. On the output side, version 2 was introduced with EPUB documents as a possible output, and over time is likely to be developed further.

Version 2 introduces a new processing layer, which relies more on the programming language Ruby' s objects (and regular expressions), than version 1 does which relies on regular expressions. The thinking behind version 1 use of regular expressions was that it made it more straightforward to switch languages for processing, as many languages support regular expressions; the thinking behind version 2 was version 1 was more complicated than it need be, and since Ruby was the language used, why not make programming more straightforward, as it would be easier to develop further.

Version 1 was removed with the introduction of the version 3 development branch


[ document manifest ]

<< previous toc next >>