Misplaced Pages

#482517

70-568: Quartz Composer is a node graph system provided as part of the Xcode development environment in macOS for processing and rendering graphical data. It is capable of making sophisticated animations for keynote or presentations and creating animated screensavers. Quartz Composer uses OpenGL (including GLSL ), OpenCL (only in Mac OS X Snow Leopard and later), OpenAL , Core Image , Core Video , JavaScript and other technologies to create an API and

140-462: A NSCoder subclass to some other format. Two main coders exist for the purpose of serializing objects to plists: Among other things, using an archiver allows for new datatypes to be encoded without changing the plist format itself and it is the preferred way for Apple to encode things like NSSets and null values. Parsing the formats do prove a bit harder, since one more layer must be followed even for some classes plists were supposed to support. Like

210-615: A Cocoa or Carbon application via supplied user interface widgets . While Quartz Composer is included with the iPhone SDK, as of December 2015 there is no way of running Quartz Compositions on iOS devices. Starting in macOS Catalina, the Quartz Composer framework has been deprecated, although it is still present for compatibility. Quartz programming through Quartz Composer works by implementing and connecting patches . Similar to routines in traditional programming languages, patches are base processing units. They execute and produce

280-650: A programming language . This same format was used by OPENSTEP . One limitation of the original NeXT property list format is that it could not represent an NSValue (number, Boolean, etc.) object. As a result, these values would have to be converted to string, and "fuzzily" recovered by the application. Another limitation is that there is no official 8-bit encoding defined. The defaults utility, introduced in OPENSTEP (1996), can be used to manipulate plist files used for storage of preferences (known as defaults in NeXTSTEP, hence

350-459: A binary file will result in a copy of the data element being placed into the XML file. Additionally the binary file has a UID type that is used to identify data items when serialized. The complete list of data that can be stored taken from the C code source file is as follows: Note the v"1?"+ note in many types. This means that the marker byte is only found in files with a format version no lower than

420-473: A by-product of execution . A simple example is a node that adds two numbers together. The inputs are the two numbers to add and the output is the sum of the two numbers. Nodes are analogous to mathematical functions of the following form. where f n o d e A {\displaystyle f_{nodeA}} is the node's computation, i n p u t s n o d e A {\displaystyle inputs_{nodeA}}

490-405: A developer tool around a simple visual programming paradigm. Apple has embedded Quartz technologies deeply into the operating system . Compositions created in Quartz Composer can be played standalone in any QuickTime -aware application (although only on Mac OS X Tiger and later), as a system Screen Saver , as an iTunes Visualizer, from inside the Quartz Composer application, or can be embedded into

560-422: A few of the largest discussions to date. Research studies tend to shed more details on these discussions and highlight more of the advantages and disadvantages of node graphs. They indicate that node graphs and visual programming are easy to understand for new users, but as the users move to more complex tasks they often need to resort to authoring textual source code . Another survey focuses on peoples beliefs on

630-480: A flowchart-based graphical programming language and could recognize handwritten letters and gestures. Alan Kay has given a number of demos of the GRAIL system , however, he was not involved with the creation of the system. Some of the more recent uses of node graph architectures started around 2005. Node graphs in this time frame start to develop paradigms to deal with complexity in the node graph. The complexity arose as

700-487: A loss function, where the loss function depends on the difference between the weights in the output layer and the expected values. Node graphs are used to visualize, configure and debug these neural network layers. The following are examples of machine learning software using node graph architecture without a graphical interface for the node graphs. The following are some examples of machine learning software using node graph architecture . Property list In

770-517: A macro. Macros can be nested and their subroutines also edited. To control the order of rendering, each renderer is assigned a layer, indicated in its upper-right corner. Layers are rendered sequentially, lowest to highest. Renderers can be enabled or disabled, essentially turning on or off that particular layer. Turning off unused layers often results in better performance, since fewer upstream patches need to be evaluated. Some patches can have subpatches, which allows for global parameter changes to just

SECTION 10

#1732851437483

840-418: A partially-compatible JSON format ( NSJSONSerialization ). In Mac OS X 10.0 , the NeXTSTEP format was deprecated , and a new XML format was introduced, with a public DTD defined by Apple . The XML format supports non-ASCII characters and storing NSValue objects (which, unlike GNUstep's ASCII property list format, Apple's ASCII property list format does not support). Since XML files, however, are not

910-530: A property list file from one format to another. It also supports converting plists to Objective-C or Swift object literals. Like the Cocoa NSPropertyListSerialization it is built on, it takes "old-style" inputs, but does not convert to this type. (The Cocoa NSSerializer from before Mac OS X 10.2 emits old-styled output.) The pl utility is introduced in Mac OS X v10.5. It takes any input and tries to generate "old-style" plists. Like

980-406: A result. For better performance, patch execution follows a lazy evaluation approach, meaning that patches are only executed when their output is needed. There are three types of patches: Consumers, Processors and External Input patches that can receive and output mouse clicks, scrolls and movements; MIDI and audio; keyboard; or other movements. A collection of patches can be melded into one, called

1050-457: A simple neural network composed of 3 layers. The 3 layers are the input layer, the hidden layer, and the output layer. The elements in each layer are weights and are connected to weights in other layers. During inference, the machine learning algorithm evaluates the weights in the output layer through a sequence of functional evaluations over the weights from previous layers. During training, the machine learning algorithm uses optimization to minimize

1120-410: A single machine, or even spanned across several machines and displays. Support for some Automator actions was added with the release of Leopard. Pierre-Olivier Latour originally developed the predecessor to Quartz Composer under the name PixelShox Studio . Node graph architecture The source code for the software application is organized into atomic functional units called nodes. This

1190-547: Is a vector of the node's input values and o u t p u t s n o d e A {\displaystyle {outputs}_{nodeA}} is a vector of the node's output values. Visually nodes are often represented by rectangles. However, this is not a convention that is followed by all applications. In the diagram above there are three nodes labeled "Video", "Add Star" and "Add Circle". Nodes often have additional parameters, that define their execution . These parameters are backed by data types in

1260-541: Is a vector of the node's output values. Visually the inputs and outputs of nodes are often represented with circles. Links transfer the values stored in data types between different nodes. They are analogous to mathematical composition. For example, if node A is feeding its outputs to node B, this can be represented mathematically as follows. where f n o d e B {\displaystyle f_{nodeB}} and f n o d e A {\displaystyle f_{nodeA}} are

1330-491: Is an additional protocol that Apple uses in their private API: One new protocol was added in version 4.0: There is no officially supported way to add additional protocols to Quartz Composer. However, there are some undocumented methods that may make this possible in the future. In addition to protocols, compositions can also conform to different runtimes where Quartz Composer is available. In Leopard, there are runtimes for Tiger (32-bit), as well as 32-bit and 64-bit versions of

1400-448: Is an ongoing effort by Eric Hosick on Twitter to collect snapshots of all node graph user interfaces in most software applications . The effort attempts to document the evolution and explosion of node graph user interfaces starting from their initial roots. This visual history is hosted on a blog page called Visual Programming Languages - Snapshots . Work leading to node graph architectures and visual programming seems to have started in

1470-609: Is defined recursively like the textual formats, with a single-byte type marker preceding some data. A form of string interning is supported via a GS-extension shouldBeCompact switch. Two relative independent plist handlers are found in GNUstep: the CFPropertyList in libs-core-base (CoreFoundation), and the NSPropertyList in libs-base (Foundation Kit). Both support the binary and XML forms used by macOS to some degree, but

SECTION 20

#1732851437483

1540-447: Is especially popular in the film and video game industries. The diagram above shows a simplified user interface for an artistic tool for editing and creating videos. The nodes are represented as rectangles and are connected to each other through curved lines ( Bézier curves ). In this software's operational model, a video sequence is being passed through the lines onto the next node, and each node performs some additional modifications to

1610-558: Is most apparent when working with text-based images, or PDFs. Version 3.0 added the ability to add annotations to areas of the composition, called notes . These notes parallel comments in other programming languages. Notes can be yellow, red, green, blue, or gray, and can overlap other notes. In Version 3.0, the concept of Composition Protocols was introduced. Protocols provide a template of required and optional inputs and outputs to qualify conforming compositions for various purposes. The following protocols are available by default: There

1680-482: Is often thought to have led the way to what is known as computer-aided design (CAD) today. In 1969, T. O. Ellis, J. F. Heafner, and W. L. Sibley published a paper concerning a Graphical Input Language (GRAIL) . Their work was related to the RAND Tablet which began with research on Sketchpad , a system where users could write computer commands directly on a tablet, conducted by Ivan Sutherland . The GRAIL system used

1750-464: Is represented as <*BY> and NSDate objects are represented as <*DYYYY-MM-DD HH:MM:SS +ZZZZ> . Binary data can also use the more efficient base64 format as <[ b64... ]> . The 8-bit problem is implicitly solved as well, as most deployments use UTF-8. GNUstep also has its own binary format, NSPropertyListGNUstepBinaryFormat , implemented in NSSerialization . This format

1820-491: Is spread across 3 file system locations: Adding compositions to the repository is as simple as adding the composition file to one of these locations. It became possible to compare compositions in Quartz Composer 3.0. This feature allows the user to compare inputs, rendered output and graph appearance of any two compositions. A developer tool called Quartz Composer Visualizer was released with Quartz Composer 3.0 that allows compositions to be rendered across multiple screens on

1890-458: Is taking place, Orange indicates a possible loss of data from conversion (Number to Index) and Red indicates a severe conversion; Image to Boolean, for example. Quartz Composer documents are called Compositions . Compositions are Binary Property Lists (though XML versions are also supported) with a filename extension .qtz and a com.apple.quartz-composer-composition UTI . Patches, their connections and their input port states are saved in

1960-426: Is typically done using classes derived from a base class for all nodes. Each node can have inputs and outputs, which are typically also implemented using classes derived from base classes for all inputs and all outputs. Outputs and inputs can refer to each other, typically by holding pointers to instances of other outputs or inputs. When a node executes its functionality, it retrieves its inputs by following

2030-408: Is well studied. One particular area of concern during node graph evaluation is cycles . When cycles are present in the node graph, the evaluation never ends as nodes are continually executed by following links. To avoid these problems many node graphs architectures restrict themselves to a subset of graphs known as directed acyclic graphs . The use of node graph architecture in software design

2100-410: The macOS , iOS , NeXTSTEP , and GNUstep programming frameworks , property list files are files that store serialized objects . Property list files use the filename extension .plist , and thus are often referred to as p-list files. Property list files are often used to store a user's settings. They are also used to store information about bundles and applications , a task served by

2170-516: The mouse , users will typically be able to: With the increasing usage of node graphs, there is currently increased attention on creating user-friendly interfaces. Often these new interfaces are being designed by user interface specialists and graphical designers. The following are some user interfaces designed by artists and designers. Many theoretical results from graph theory apply to node graphs, especially with regards to topology . This subject area where Nodes are linked together to form graphs

Quartz Composer - Misplaced Pages Continue

2240-410: The pointers stored in its inputs to retrieve data output by other nodes. The node then executes its operation on these inputs to produce its own outputs. The ability to link nodes together in this way allows complex tasks or problems to be broken down into atomic nodal units that are easier to understand. The user interface of the software application will often visually display the node graph to

2310-455: The resource fork in the old Mac OS. Property lists are also used for localization strings for development. These files use the .strings or .stringsdict extensions. The former is a "reduced" old-style plist containing only one dictionary without the braces (see propertyListFromStringsFileFormat ), while the latter is a fully-fledged plist. Xcode also uses a .pbxproj extension for old-style plists used as project files. Since

2380-456: The "1?" magic number. The precise way to parse them is more nebulous than the way to parse legacy types, since the CFBinaryPlist implementation only handles version "0?". In practice, these types are never encountered, since NSKeyedArchiver is already capable of capturing these information. A table of offsets follow the object table, which is then followed by a trailer containing information on

2450-425: The 1960s, in the area known as "man-machine communications". In William Robert Sutherland's MIT thesis (1966) "Online Graphical Specification of Procedures" , he describes and analyses topics around a 2D pictorial language. This is one of the first investigations in dataflow-based workflows or programs. Since then his thesis has been used as "prior art" in order to quash lawsuits about dataflow ideas today. His work

2520-479: The GNUstep version, it appears to use the description property of Foundation types found in plists, which Apple has specified to produce valid old-style plists. In terms of the internals, Apple provides an open source parser for old style, XML, and binary formats in their C Core Foundation code as CFPropertyList. However, all the utilities and most parts of the system use the closed-source NSPropertyList parser from

2590-505: The JSON format does not support. Conversely, JSON permits null values while property lists do not support explicit nulls. The old defaults tool from NeXTSTEP remains available. The /usr/libexec/PlistBuddy command provides an interactive plist editor. It can also be scripted. The plutil utility (introduced in Mac OS X 10.2 ) can be used to check the syntax of property lists, or convert

2660-598: The Leopard Quartz Composer runtime. The editor can also indicate used patches that are unsafe, or unavailable in Tiger to aid in making compatible compositions. A System-wide Composition Repository is available as of Version 3.0. This allows applications to share and make use of common compositions for effects and processing. It is also possible for applications to query the repository for compositions that match certain criteria, such as protocol conformance. The Repository

2730-524: The Obj-C Foundation Kit. The Swift reimplementation is open source, but is not guaranteed to be identical. XML and JSON property lists are hand-editable in any text editor. Additionally, Apple provides support in Xcode for editing property lists in a hierarchical viewer/editor that can handle plists formatted in binary or XML , but not JSON . As of Mac OS X 10.4 , Apple provides an AppleScript interface for reading property list files through

2800-629: The System Events application. As of Mac OS X 10.5 , Apple provides an AppleScript interface for editing, creating and writing property list files as well. For the XML format, the tags, related Foundation classes and CoreFoundation types, and data storage formats are as follows: The binary file format is documented in a comment block in the Core Foundation C code source file ( CF/CFBinaryPList.c ) for Apple's open sourced implementation of binary plists in its Foundation library. Apple describes

2870-412: The XML or JSON file formats. The array, set and dictionary binary types are made up of pointers - the objref and keyref entries - that index into an object table in the file. This means that binary plists can capture the fact that - for example - a separate array and dictionary serialized into a file both have the same data element stored in them. This cannot be captured in an XML file. Converting such

Quartz Composer - Misplaced Pages Continue

2940-561: The binary format which also has an object table, it is possible to create circular references in NSKeyedArchiver . Since there is not a UID data type in XML, the integers are stored in a dictionary under the key "CF$ UID". Apple publishes an open-source NSKeyedArchiver in Swift Corelibs Foundation; like the closed-source Apple Foundation, it restricts output formats to binary and XML only. It also has some test cases showing

3010-535: The cognitive effects of visual programming, in which they found that professional programmers are the most skeptical of visual programming. Other studies have shown in psychological experiments that visual programming can have significant positive effects on performance in cognitive tasks. A node graph in the context of software architecture refers to an organization of software functionality into atomic units known as nodes, and where nodes can be connected to each other via links. The manipulation of nodes and links in

3080-991: The composition file. Images can be stored inside a composition as well, making for self-contained compositions with embedded graphics. By dragging a movie file into the Quartz Composer editor, a reference to the movie file is created, providing a changing image that can be connected to a renderer. Compositions also store metadata such as composition author, copyright and description. The user can also add arbitrary metadata items, if desired. Many image formats are supported, including JPEG , JPEG2000 , GIF , PNG , TIFF , TGA , OpenEXR , BMP , ICO , PDF , PICT , ICNS and some raw digital camera types. Images are maintained in their native form for as long as possible before rasterizing for display. This means that Quartz Composer will keep vector images as vectors when cropping, scaling, rotating, or translating which allows it to work with very large logical image dimensions without consuming large amounts of memory or processing time. Such functionality

3150-408: The data represented by property lists is somewhat abstract , the underlying file format can be implemented many ways. Namely, NeXTSTEP used one format to represent a property list, and the subsequent GNUstep and macOS frameworks introduced differing formats. Under NeXTSTEP , property lists were designed to be human-readable and edited by hand, serialized to ASCII in a syntax somewhat like

3220-424: The file is in the XML or JSON format with care a text editor such as Notepad++ can be used. Apple ships a plutil.exe within its "Apple Application Support" package (which is part of iTunes ), and it is identical to its macOS counterpart. Introduced in 2006 and first released with NetBSD#4.0 (2007) is a proplib library, which can be used for serialising data between the kernel and userland. It implements part of

3290-544: The files are typically binary files, although some applications may generate PLIST files in the other formats. On Windows the Apple plist files are stored in the user's home directory under %USERPROFILE%\AppData\Roaming\Apple Computer . These plist files on Windows typically store preferences and other information, rather than using the Windows registry. Options for editing PLIST files on Windows are not as extensive as on macOS. If

3360-402: The group. This leads to a hierarchy where smaller graphs are embedded in group nodes. The following are examples of group nodes which are used to group a subset of connected nodes and to help simplify the graph. Software applications using node graph architecture will typically expose the node graph visually or graphically to the user, allowing the user to make changes to the node graph. Using

3430-404: The implementation as opaque in its plist(5) manual page documentation, which means that reliance on the format is discouraged. In the binary file format the magic number (the first few bytes of the file which indicate that it's a valid plist file) is the text bplist , followed by two bytes indicating the version of the format. The binary file can store some information that cannot be captured in

3500-434: The included subpatches. This is useful for lighting, 3D transformation and GLSL shaders, among other things. Subpatch support is indicated by square corners on a patch, rather than the typical rounded corners. With Version 3.0, it became possible to turn compositions into Virtual Patches. These allow the user to reuse functionality without having to store duplicate copies in each composition. The Quartz Composer Editor allows

3570-589: The internal structure and operation of the software. Node graphs are a subset of the broader class of visual programming languages . Node graphs allow you to design programs in a visual and structured way instead of through the authoring of source code . In the film and video game industries node graphs are synonymous with visual programming. There is currently some debate on the power, abstraction, and need of node graphs and visual programming languages. This remains an active area of debate with new discussions occurring in open forums to this day. The following are

SECTION 50

#1732851437483

3640-405: The latter is a lot more complete. For example, the two GNUstep-specific formats are only handled in the latter. GNUstep provides a set of plist command-line tools based on NSPropertyList , including a version of pl and defaults . While macOS can also read the NeXTSTEP format, Apple sets it aside in favor of two new formats of its own, one XML-based and the other binary. Apple also has

3710-459: The main adopters of this software design with the majority of tools using node graph architecture. To this day, there is some debate as to the benefits of visual programming and node graph architecture. Advocates highlight how the abstraction that node graphs provide makes the tool easier to use. Critics highlight how visual programming is too restrictive and how they must resort to modifying source code or scripts to accomplish their tasks. There

3780-494: The most space-efficient means of storage, Mac OS X 10.2 introduced a new format where property list files are stored as binary files. Starting with Mac OS X 10.4 , this is the default format for preference files. In Mac OS X 10.7 , support for reading and writing files in JSON format was introduced. JSON and property lists are not fully compatible with each other, though. For example, property lists have native date and data types, which

3850-429: The name) on the command line via their preferences domain, and this utility can be used to edit arbitrary plist files. This utility superseded three older commands (dread, dwrite, and dremove). GNUstep adopts the NeXTSTEP format, with additions for representing NSValue and NSDate data types. The new typed entries have the form <* T ...> , where T is a one-letter type code. For example, an NSValue of Boolean YES

3920-481: The node can add an extra fixed number onto the sum. Visually the node's parameters are often exposed after the user clicks on the node. This helps to reduce visually cluttering the node graph. In the diagram above we see a parameter window opening up beside the "Add Star" node. Nodes often have inputs and outputs, as discussed above. Inputs and outputs are backed by data types in the node's source code . Inputs and outputs are crucial to storing values before and after

3990-493: The node graph can be often be accomplished through a programmable API or through a visual interface by using the mouse . In the diagram above, the node graph appears on the right-hand side. In modern-day usage, the term "node graph" is an open compound word. However, in older software it was referred to as a "nodegraph", a closed compound word. Nodes perform some type of computation. They encapsulate this executable functionality and will often take inputs and produce outputs as

4060-417: The node graph. The following are some examples: The most important node type for managing complexity is the group node. This node type does not execute software code in the same as other nodes. This node simply groups a subset of connected nodes together and manages the inputs and outputs into or out of the group. This hides complexity inside of the group nodes and limits their coupling with other nodes outside

4130-528: The node's execution . Mathematically the node's inputs and outputs are analogous to input and output values of functions. where f n o d e A {\displaystyle f_{nodeA}} is the node's computation, i n p u t s n o d e A {\displaystyle inputs_{nodeA}} is a vector of the node's input values and o u t p u t s n o d e A {\displaystyle {outputs}_{nodeA}}

4200-412: The node's source code . Mathematically they can be thought of as additional input values to the node's compute function. The only difference is that these values are controlled directly by the user instead of being output by another node as a by-product of its execution . For example, in the simple example above regarding a node that adds two numbers, we can introduce a bias parameter on the node so that

4270-676: The number of nodes and links in the graph increased. One of the main ideas dealing with complexity was the concept of a group or package node which hid nodes inside of itself, only exposing the inputs and outputs of the group. In the paper Hierarchical Small Worlds in Software Architecture author Sergi Valverde argues that most large software systems are built in a modular and hierarchical fashion, and that node graphs can be used to analyze large software systems. Many other software analysis papers often use node graphs to analyze large software systems suggesting that node graphs are good models of

SECTION 60

#1732851437483

4340-521: The operations performed by node B and node A, i n p u t s n o d e A {\displaystyle inputs_{nodeA}} is a vector of the node A's input values and o u t p u t s n o d e B {\displaystyle outputs_{nodeB}} is a vector of the node B's output values. The type of a node indicates which compute operation it will perform when executed . There are often many different node types participating in

4410-585: The release of Leopard. It became possible to transmit data and synchronize over a network interface and it also added support for Open Sound Control transmission and reception. Also new in Version 3.0 was the possibility to write custom patch plugins using an Xcode template, and the notion of a " safe mode " where plugins and other unsafe patches fail to load. This prevents malicious compositions from performing dangerous or insecure operations. Custom patches using Apple's Xcode template are always considered unsafe. It

4480-547: The results of serialization. GNUstep also has a compatible implementation, which does not limit output formats. There is not a single, standardized path language for property lists like XPath does for XML, but informal conventions used by various programs exist. Neither format is able to express a key with the separator character in it. Although best known on Apple or Darwin systems, including iOS and macOS, plist files are also present on Windows computers when Apple software, such as iTunes or Safari are installed. On Windows,

4550-448: The size and location of the two tables. Since property lists do not capture all the information and data types required to describe an arbitrary object, an extra layer of encoding and decoding is often done. The OpenStep specification abstracts the operation of serializing any NSObject under the NSCoding protocol. Any class implementing this protocol can have its instances serialized by

4620-410: The user to save a "flattened" copy (with the virtual patches fully expanded inside), for easy distribution. Version 4.0 extended this functionality even more and automatically includes "flattened" copies of virtual patches for use as a fallback if the desired virtual patch isn't installed on the host system. This greatly simplifies composition distribution. Network functionality was greatly improved with

4690-506: The user. This is often accomplished by using the GPU to perform the rendering which is subsequently displayed on the desktop to the user. Common APIs for the GPU are OpenGL and DirectX . Nodes are often drawn as rectangles, and connections between nodes are drawn with lines or splines. The use of node graph architecture started in the 1960s. Today the use of node graphs has exploded. The fields of graphics, games , and machine learning are

4760-400: The video sequence. In this example one video is translated in 2D, another is pixelated, and finally, both streams are merged. The following are some examples of software using node graph architecture in the film and video game industries. The use of node graph architecture in software design has recently become very popular in machine learning applications. The diagram above shows

4830-516: Was added as a hidden option in version 4.0, allowing for antialiasing inside the QC Editor, though it only works on GPU's that support MSAA. Data inside QC can be one of the following types: Two additional types were introduced in version 4.0: Data can usually be converted to other types transparently. In Quartz Composer 3.0, the connections between patches change color to indicate conversions that are taking place. Yellow connections mean no conversion

4900-631: Was possible to develop custom patch plugins for Version 2.0, but the API was undocumented and private and was never supported by Apple. Eventually, templates were released to simplify this procedure. In the Quartz Composer editor, holding the option key while selecting "Preferences..." from the menu adds 3 additional tabs of options for the user to configure. These options include System settings, Editor settings and QuickTime integration settings. Notable options include expanded tooltips, software rendering and uncapped framerate rendering. Multisample antialiasing (MSAA)

#482517