(PDOC-23) Document JSON structure format

README.md

@@ -90,6 +90,20 @@ To document specific files:
 $ puppet strings some_manifest.pp [another_if_you_feel_like_it.rb]
 ```
 
+Strings can also emit the generated documentation as JSON:
+
+```
+$ puppet strings some_manifest.pp --emit-json documentation.json
+```
+
+It can also print the JSON to stdout:
+
+```
+$ puppet strings some_manifest.pp --emit-json-stdout
+```
+
+The schema for the JSON which Strings emits is [well documented](json_dom.md).
+
 Processing is delegated to the `yardoc` tool so some options listed in `yard
 help doc` are available.  However, Puppet Faces do not support passing
 arbitrary options through a face so these options must be specified in a

json_dom.md

@@ -0,0 +1,121 @@
+The Strings JSON Interchange Schema
+===================================
+
+Strings has two flags used to emit json.
+* `--emit-json $FILE` Saves json to a file.
+* `--emit-json-stdout` Prints json on stdout.
+
+Top Level Structure
+-------------------
+
+The json outputted by strings is a single object which has 4 keys representing
+the different types of Puppet code and extension functions Strings reads. The
+value for each key is a list of json objects representing each puppet class,
+function, etc.
+Here is an example of the top level structure:
+
+```json
+{
+
+"defined_types": [...],
+
+"puppet_classes": [...],
+
+"puppet_functions": [...],
+
+"puppet_types": [...],
+
+"puppet_providers": [...]
+
+}
+```
+
+Defined Types
+-------------
+
+Each defined type or puppet class object has the following properties and values:
+
+* `name`: A string representing the name of the object.
+* `file`: The file the object came from. A string.
+* `line`: The line in the file the object came from. A number.
+* `docstring`: A string. The docstring describing the object.
+* `examples`: A list of strings representing the content of the examples in the
+   docstring.
+* `signatures`: A list of function signatures which may be supported by the
+  object. Each function signature is a json object whose keys are the
+  parameter names, and whose values are the types those parameters may take.
+  This is extracted from the code itself.
+* `parameters`: An object whose keys are the parameter names and whose values
+  are the parameter's types or null if it has no types. This is extracted from
+  the docstring.
+
+
+Puppet Functions
+----------------
+
+Both puppet 4x and 3x functions are represented as json objects kept in the
+`puppet_functions` list. Puppet 4x functions have every property that 3x
+functions have, as well as a few extras.
+
+Puppet 3x functions have:
+
+* `name`: A string representing the name of the 
+* `file`: The file the object came from. A string.
+* `line`: The line in the file the object came from. A number.
+* `docstring`: A string. The docstring describing our object.
+* `function_api_version`: the number 3.
+* `documented_params`: A object whose keys are the parameters which were
+* documented and whose values are the types they may take, or null.
+* `examples`: A list of strings representing the content of the examples in the
+   docstring.
+
+Puppet 4x functions have everything 3x functions do as well as:
+
+* The `function_api_version` is the number 4, not 3 (surprise!).
+* `signatures`: A list of function signatures which may be supported by the
+  object. Each function signature is a json object whose keys are the parameter
+  names, and whose values are the types those parameters may take. This is
+  extracted from the code itself.
+
+Puppet Types
+------------
+
+Each puppet type object has the following properties and values:
+
+* `name`: A string representing the name of the object
+* `file`: The file the object came from. A string.
+* `line`: The line in the file the object came from. A number.
+* `docstring`: A string. The docstring describing our object.
+* `examples`: A list of strings representing the content of the examples in the
+   docstring.
+* `parameters`: A list of objects with the following shape:
+    * `allowed_vales`: a list of strings representing the allowed values.
+    * `default`: a string or null.
+    * `docstring`: The docstring.
+    * `name`: the parameter name.
+* `properties`: A list of objects with a shape very similar to parameters but
+  also including:
+    * `namevar`: A boolean.
+* `features`: A list of objects representing possible features. They have the
+  following shape:
+    * `docstring`: The description of the feature.
+    * `methods`: null or a list of the available methods as strings.
+    * `name`: The feature's name.
+
+Puppet Providers
+----------------
+Each puppet provider object has the following properties and values:
+
+* `name`: A string representing the name of the object
+* `file`: The file the object came from. A string.
+* `line`: The line in the file the object came from. A number.
+* `docstring`: A string. The docstring describing the object.
+* `examples`: A list of strings representing the content of the examples in the
+   docstring.
+* `commands`: A list of the names of the commands available.
+* `confines`: An object whose keys are the confine keys and whose values are
+  the confine values.
+* `defaults`: Similar to above.
+* `features`: A list of strings representing the features this provider
+  supports.
+* `type_name`: The type this provider accompanies.

lib/puppet/face/strings.rb

@@ -22,6 +22,13 @@ Puppet::Face.define(:strings, '0.0.1') do
   action(:yardoc) do
     default
 
+    option "--emit-json-stdout" do
+      summary "Print json representation of the documentation to stdout"
+    end
+    option "--emit-json FILE" do
+      summary "Write json representation of the documentation to FILE"
+    end
+
     summary "Generate YARD documentation from files."
     arguments "[manifest_file.pp ...]"
 

lib/puppet_x/puppetlabs/strings/util.rb

@@ -14,7 +14,8 @@ module PuppetX::PuppetLabs::Strings::Util
     # YARD options are passed to it. The best way to approach this problem is
     # by using the `.yardopts` file. YARD will autoload any options placed in
     # that file.
-    args.pop
+    options = args.pop
+    YARD::Config.options = YARD::Config.options.merge(options)
 
     # For now, assume the remaining positional args are a list of manifest
     # and ruby files to parse.

lib/puppet_x/puppetlabs/strings/yard/code_objects/defined_type_object.rb

@@ -25,6 +25,9 @@ class PuppetX::PuppetLabs::Strings::YARD::CodeObjects::DefinedTypeObject < Puppe
           }
         end
       end,
+      "examples"              => self.tags.map do |tag|
+          tag.text if tag.tag_name == 'example'
+      end.compact,
     }.to_json(*a)
   end
 end

lib/puppet_x/puppetlabs/strings/yard/code_objects/host_class_object.rb

@@ -3,22 +3,6 @@ class PuppetX::PuppetLabs::Strings::YARD::CodeObjects::HostClassObject < PuppetX
   # @return [HostClassObject, Proxy, nil]
   attr_accessor :parent_class
 
-#  def to_json(*a)
-#    {
-#      "name"             => @name,
-#      "file"             => file,
-#      "line"             => line,
-#      "docstring"        => Puppet::Util::Docs.scrub(@docstring),
-#      "parameters"       => Hash[@parameters],
-#      "signatures"       => @type_info.map do |key, value|
-#          {
-#            "name" => key,
-#            "type" => value,
-#          }
-#      end,
-#    }.to_json(*a)
-#  end
-
   # NOTE: `include_mods` is never used as it makes no sense for Puppet, but
   # this is called by `YARD::Registry` and it will pass a parameter.
   def inheritance_tree(include_mods = false)

lib/puppet_x/puppetlabs/strings/yard/code_objects/method_object.rb

@@ -16,8 +16,11 @@ class YARD::CodeObjects::MethodObject
         "name"                  => @name,
         "file"                  => file,
         "line"                  => line,
-        "puppet_version"      => 4,
+        "function_api_version"  => 4,
         "docstring"             => Puppet::Util::Docs.scrub(@docstring),
+        "examples"              => self.tags.map do |tag|
+            tag.text if tag.tag_name == 'example'
+          end.compact,
         "documented_params"     => @parameters.map do |tuple|
           {
             "name" => tuple[0],
@@ -38,7 +41,7 @@ class YARD::CodeObjects::MethodObject
         "name"                  => @name,
         "file"                  => file,
         "line"                  => line,
-        "puppet_version"      => 3,
+        "function_api_version"  => 3,
         "docstring"             => Puppet::Util::Docs.scrub(@docstring),
         "documented_params"     => @parameters.map do |tuple|
           {
@@ -46,6 +49,9 @@ class YARD::CodeObjects::MethodObject
             "type" => tuple[1],
           }
         end,
+        "examples"              => self.tags.map do |tag|
+            tag.text if tag.tag_name == 'example'
+          end.compact,
       }.to_json(*a)
     else
       super

lib/puppet_x/puppetlabs/strings/yard/code_objects/provider_object.rb

@@ -14,6 +14,9 @@ class PuppetX::PuppetLabs::Strings::YARD::CodeObjects::ProviderObject < PuppetX:
       "confines"         => @confines,
       "defaults"         => @defaults,
       "features"         => @features,
+      "examples"              => self.tags.map do |tag|
+          tag.text if tag.tag_name == 'example'
+      end.compact,
     }.to_json(*a)
   end
 

lib/puppet_x/puppetlabs/strings/yard/code_objects/puppet_namespace_object.rb

@@ -17,6 +17,9 @@ class PuppetX::PuppetLabs::Strings::YARD::CodeObjects::PuppetNamespaceObject < Y
       "file"             => file,
       "line"             => line,
       "docstring"        => @docstring,
+      "examples"              => self.tags.map do |tag|
+          tag.text if tag.tag_name == 'example'
+      end.compact,
     }.to_json(*a)
   end
 

lib/puppet_x/puppetlabs/strings/yard/code_objects/type_object.rb

@@ -18,6 +18,9 @@ class PuppetX::PuppetLabs::Strings::YARD::CodeObjects::TypeObject < PuppetX::Pup
           "name"           => obj[:name],
         }
       end,
+      "examples"              => self.tags.map do |tag|
+          tag.text if tag.tag_name == 'example'
+      end.compact,
       "properties"         => @property_details.map do |obj|
         {
           "allowed_values" => obj[:allowed_values] ? obj[:allowed_values].flatten : [],

lib/puppet_x/puppetlabs/strings/yard/json_registry_store.rb

@@ -20,6 +20,7 @@ module YARD
     # @param obj [Hash] A hash representing the registry or part of the
     # registry.
     def serialize_output_schema(obj)
+
         schema = {
           :puppet_functions => [],
           :puppet_providers => [],
@@ -61,17 +62,23 @@ module YARD
       def initialize o
         super
         @options = {
-          :basepath => 'doc',
+          :basepath => '.',
           :extension => 'json',
         }
         @extension = 'json'
-        @basepath = 'doc'
+        @basepath = '.'
       end
       def serialize(data)
-        path = File.join(basepath, "registry_dump.#{extension}")
+
+        if YARD::Config.options[:emit_json]
+          path = File.join(basepath, YARD::Config.options[:emit_json])
           log.debug "Serializing json to #{path}"
           File.open!(path, "wb") {|f| f.write data }
         end
+        if YARD::Config.options[:emit_json_stdout]
+          puts data
+        end
+      end
     end
   end