tag:blogger.com,1999:blog-58168422057277216942015-03-18T22:45:37.796-07:00Open Source Business and Open For BusinessOpen Source Business IS Open For BusinessDavid Joneshttps://plus.google.com/100501146496217472449noreply@blogger.comBlogger34125tag:blogger.com,1999:blog-5816842205727721694.post-87379442589527703452015-03-18T22:43:00.001-07:002015-03-18T22:45:37.857-07:00Using Elasticsearch and Moqui for Analytics<h3>Introduction to Elasticsearch in Moqui</h3><br />Elasticsearch has been used in Moqui for years for search purposes, finding things like tasks, products, and wiki pages. It is a great tool for finding data quickly from a wide variety of sources indexed using Apache Lucene for flexible full-text searching.<br /><br />The Data Document feature in Moqui makes it easy to define a JSON document (or nested Map/List structure) derived from relational database records. The Moqui Data Feed feature provides a real-time link between Data Documents and tools that use them, including the service built into Moqui that receives documents from a feed to index them for searching and other purposes. This can also be used to send data to other systems, trigger emails, and so on.<br /><br />Elasticsearch is much more than just a full-text search tool. The faceted searches are helpful when searching structured data, but indexing by different fields within a document can be used for so much more. To get an idea of the scope and usefulness of Elasticsearch look at the home page for it:<br /><br /><a href="https://www.elastic.co/products/elasticsearch">https://www.elastic.co/products/elasticsearch</a><br /><br /><h3>Analytics Capabilities and Easy Use in Moqui</h3><br />The analytics capabilities of ES combine the distributed big-data and faceted search capabilities with a wide variety of tools for bucketing and aggregating data. For a list of the aggregation functions available, and a summary of the aggregation feature in general, see:<br /><br /><a href="http://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html">http://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html</a><br /><br />It has basic numeric functions for "metrics" like min, max, sum, and avg plus more advanced ones like percentiles, histograms, geographic areas, extracting significant terms, and even scripts for custom metrics. These can be calculated over sets of documents split into "buckets" distinguished by terms, ranges of values, and more. Filters can be applied to the documents included at the top level and in any bucket.<br /><br />For a more general idea of aggregations they are covered in one part of the book "Elasticsearch: The Definitive Guide" which is available online here:<br /><br /><a href="http://www.elastic.co/guide/en/elasticsearch/guide/current/aggregations.html">http://www.elastic.co/guide/en/elasticsearch/guide/current/aggregations.html</a><br /><br />For analytics in Moqui this provides an ideal tool for persistence of analytic data derived from operational data. The analytic data can be easily searched with results bucketed and metrics calculated as needed. Feeding data to this analytic data store is easy in Moqui by defining Data Documents that cover entity fields you want to report on, plus a real-time Data Feed to send the documents to Elasticsearch for indexing as records in the operational database are updated.<br /><br /><h3>Real World Examples from Mantle Business Artifacts</h3><br />For some real world examples of Data Documents useful for analytics see the MantleDocumentInventoryData.xml file in Mantle Business Artifacts:<br /><br /><a href="https://github.com/moqui/mantle/blob/master/mantle-udm/data/MantleDocumentInventoryData.xml">https://github.com/moqui/mantle/blob/master/mantle-udm/data/MantleDocumentInventoryData.xml</a><br /><br />This file has documents for inventory data, both current and projected, based on inventory asset records, sales order items, production estimates, and production run consume/produce estimates and actuals. Over time this will be extended to include shipments, purchase orders, and other inventory-relevant data structures. Most of these documents include data/time fields that can be used to limit projections to a certain period in the future and along with statusId fields exclude data not useful for future projections. Documents from the past can be used for reporting on the history of inventory and statistics about incoming and outgoing items.<br /><br />A big part of the efficiency in reporting comes from pulling together data from various parts of the operational database into a single document that is persisted and indexed in the analytic data store (in this case Elasticsearch). That is what these Data Document definitions do. They pull together the needed fields from various tables into a single JSON document that can be used for efficient filtered and bucketed searches with metrics calculated on the fly.<br /><br />Note (if you want to run this stuff) that to actually create and index documents from these Data Document definitions you must uncomment the DataFeed defined at the bottom of this file, and do so before the data load (i.e. 'gradle load').<br /><br />The service in Moqui to do analytics queries is org.moqui.impl.EntityServices.search#CountBySource which is designed to take a "source" parameter with the details of the search to do in Elasticsearch. Here is an example of a source Map in Groovy syntax:<br /><blockquote class="tr_bq"><span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">&nbsp; &nbsp; andList = [[terms:[productId:productIdList]], [not:[terms: &nbsp;[partStatusId: &nbsp; ['OrderCompleted','OrderRejected','OrderCancelled']]]]]<br />&nbsp; &nbsp; if (facilityId) andList.add([term:[facilityId:facilityId]])<br />&nbsp; &nbsp; if (estThruDateStr) andList.add([or: [[missing:[field:'requiredByDate']], [range:[requiredByDate:[lte:estThruDateStr]]]]])<br />&nbsp; &nbsp; searchSourceMap = [<br />&nbsp; &nbsp; &nbsp; &nbsp; size: maxResults, query: [filtered: [filter: [and: andList] ]],<br />&nbsp; &nbsp; &nbsp; &nbsp; aggregations: [<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; products: [<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; terms: [field: 'productId', size: maxResults],<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; aggregations: [<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; orderQuantitySum: [sum: [field:'orderQuantity']],<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; quantityReservedSum: [sum: [field:'quantityReserved']],<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; quantityNotAvailableSum: [sum: [field:'quantityNotAvailable']],<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; quantityNotIssuedSum: [sum: [field:'quantityNotIssued']]<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ]<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ]<br />&nbsp; &nbsp; &nbsp; &nbsp; ]<br />&nbsp; &nbsp; ]</span></blockquote>This is from the get#InventoryProjectedInfo service in the InventoryReportServices.xml file in Mantle:<br /><br /><a href="https://github.com/moqui/mantle/blob/master/mantle-usl/service/mantle/product/InventoryReportServices.xml">https://github.com/moqui/mantle/blob/master/mantle-usl/service/mantle/product/InventoryReportServices.xml</a><br /><br />This source Map in Groovy syntax follows the same structure as JSON documents sent to Elasticsearch through its REST API, and is generally easier than the Java API to use plus more closely aligns with the ES documentation examples.<br /><br />This example is meant to get sums of various values for order items that are associated with order parts that still open and that may or may not be reserved. This is useful to see upcoming orders that will effect inventory at some point in the future.<br /><br /><h3>Pitfalls and Quirks of Elasticsearch for Analytics</h3><br />One potential issue with Elasticsearch for analytic use is the default result size of 10. When using aggregations with buckets the general "size" parameter passed in to the Java API or REST request parameter does NOT apply to the buckets. If you want more than 10 buckets in an aggregation you must specify a larger size in the bucket's "terms" Map. In the example above the terms Map is [field: 'productId', size: maxResults] using the maxResults field to allow for a higher value (defaults to 1000for the get#InventoryProjectedInfo service).<br /><br />Another big pitfall is the data type of document fields. If you try to do a sum, avg, etc on a string type field you will get an error, you need a number type field instead. Here is a reference for the core data types available in Elasticsearch:<br /><br /><a href="http://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-core-types.html">http://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-core-types.html</a><br /><br />The type of each field is specified using a "mapping" for a document type. In the most recent Moqui Framework code it automatically checks to see if an index exists and if not it creates the index and puts all the document type mappings. These type mappings are generated based on the Data Document definition and the entity field type for each field included in it. If you want to see what these mappings look like you can get them from Elasticsearch after the index has been used at least once, to index a document or to search, using a curl command like this:<br /><br /><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">curl -XGET 'http://localhost:9200/default__mantle_inventory/_mapping?pretty=true'</span><br /><br />Note that there is a separate set of indexes for each tenant in Moqui, which is prepended to the index name and separated with a double underscore. The entire extended index name is lower-cased as Elasticsearch only supports lower case index names. This is why there is a "default__" at the beginning of the index name for the DEFAULT tenant in Moqui.<br /><br />In addition to the numeric field types another important thing Moqui does in the default document mappings is to set all ID fields (entity field types id and id-long) to have index: not_analyzed. This tells Elasticsearch to use and index the literal ID value instead of analyzing it for search terms which also changes the value to all lower case (which is how ES handles case-insensitive searches). Without this fields like productId, partStatusId, facilityId, etc would be lower-cased and possibly split into multiple search terms if they contain certain characters. This seriously messes up both term/terms filters and the ID values that come back for the bucket keys.<br /><br />There are also some Data Document and Feed quirks in Moqui to be aware of. The main one is that for the real-time push feed (triggered by changes to entity records through the Entity Facade) there must be a reverse-relationship defined for each forward relationship in the field path in a data document field. For example consider this field path in the MantleInventoryOrderItem Data Document:<br /><br /><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">mantle.order.OrderPart:Vendor#mantle.party.PartyRole:roleTypeId</span><br /><br />This uses the relationship "Vendor#mantle.party.PartyRole" from OrderPart to PartyRole. For the real-time push we need the reverse relationship, i.e. one from PartyRole back to OrderPart which can be seen in the "Vendor#mantle.order.OrderPart" relationship on the PartyRole entity.<br /><br /><h3>Conclusion</h3><br />For more complex analytics, especially that involve data from many places in an operational database with a variety of constraints and with a variety of metrics applied, it is common to have a separate data store that is structured for analytics and the particular ways the data will be used.<br /><br />Star schemas in relational databases are a nice approach to this, but still limited by the flat relational structure. With hierarchical JSON documents you can include "dimensions" in "fact" documents for more efficient querying/searching while still remaining flexible. You can still have separate documents that are "joined" in a search, but because documents in the database are only loosely typed and structured and because they are hierarchical you are not limited to this.<br /><br />Relational SQL databases are also generally more difficult to scale for the large amount of data needed when indexing a large number fields across very large numbers of records.<br /><br />These are some of the reasons that NoSQL and more generally non-relational databases are becoming more popular for analytics. When not so concerned with data consistency and referential integrity, and more generally the ACID properties, sharding and otherwise distributing data across various physical machines is MUCH easier. This is great for reporting in real time on large data sets as the data can be processed on each server and then the aggregated results combined from the various servers to get the final results.<br /><br />Elasticsearch is a nice tool for this. It features much of the scalability and flexibility of document databases like MongoDB but is WAY easier to deploy in Java applications and usable in so many ways. ES can be run embedded in a Java application, as it is by default in Moqui Framework, or it can be distributed across hundreds of servers coordinating in a cluster with automatic discovery, sharding, and so on. Elasticsearch also does a pretty good job of caching frequent searches, and has all sorts of tools and configuration options for tuning searches, aggregations, and caching.<br /><br />Elasticsearch also goes beyond what most document databases provide in terms of the variety of aggregations it can run on the document data. This provides, in one package, what is only available with analytics tools built on various popular document databases.<br /><br />There is also an ad-hoc and prepared reporting tool for Elasticsearch called Kibana. It is already pretty impressive and is still being improved significantly. Unfortunately, it isn't all that easy to embed (for deployment) in a Java application but it is pretty easy to run on its own. For details see:<br /><br /><a href="https://www.elastic.co/products/kibana">https://www.elastic.co/products/kibana</a><br /><br />The Data Document and Data Feed features in Moqui Framework were designed for deriving data from the relational database to generate JSON documents. Initially the intent was to use them for searching and triggering things like sending email, but the general concept applies very well as an easy way to configure what will be pushed from the operational database to an analytic data store.<br /><br />In short this combination of tools, already available in Moqui Framework (currently unreleased but available in the GitHub repositories), provide an excellent foundation for flexible, high scale reporting and analytics. It is easy to transform and feed data from the operational database to the analytic data store, and a wide variety of operations can be performed with high efficiency on this data.<br /><br />How cool is this? With the combination of Moqui Framework and Elasticsearch you can put together complex reports that operate on huge volumes of data with very little effort.<br /><div><br /></div>David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com0tag:blogger.com,1999:blog-5816842205727721694.post-39011213156443043452014-09-23T21:13:00.001-07:002014-09-23T21:56:36.660-07:00Making Apps with Moqui - Book Now AvailableMy latest book is now available on Amazon.com:<br /><br /><a href="http://www.amazon.com/Making-Apps-Moqui-Enterprise-Applications/dp/0692267050/">http://www.amazon.com/Making-Apps-Moqui-Enterprise-Applications/dp/0692267050/</a><br /><br />Making Apps with Moqui is the official documentation for Moqui Framework and includes a comprehensive summary of Mantle Business Artifacts. Starting with basic concepts and a tutorial to try things rights away, it builds to complete examples of end-to-end business processes including procure to pay, order to cash, and work plan to cash.<br /><br /> The framework topics cover data and service tier tools, user and system interfaces, security, and performance. With dozens of diagrams and screen shots, and thousands of lines of code and configuration examples, this book gives you ideas of what you can do with Moqui Framework and shows you how too. This includes things as simple as defining your data model with entities to more advanced things like building hierarchical data documents based on entity data and feeding them to other systems or indexing and searching the documents through simple configuration.<br /><br /> Learn how to easily build remote and local services that handle validation, security, transaction management, and much more. Build screens quickly with a wide variety of dynamic widgets and forms styled any way you wish, or even define your own widgets to use consistently across your applications. Handle large scale and milt-tenant systems. Track your application use and performance. Implicitly handle multiple languages, currencies and other localization details. Control access to resources across all tiers through flexible authc and authz configuration. <br /><br />Written by the founder of Moqui and Mantle, and an enterprise application architect with 15 years of open source and commercial experience, this book provides the most accurate and useful information available for building modern enterprise applications with some of the best open source tools and technologies.<br /><br />NOTE: this book is also available for free download from <a href="http://www.moqui.org/">http://www.moqui.org</a>.<br /><div><br /></div>David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com1tag:blogger.com,1999:blog-5816842205727721694.post-45591695998563373412014-06-19T10:02:00.000-07:002014-06-19T10:02:27.981-07:00Why I Work Without Pay on Free Software: Flow and Autotelic Creative Work<h3>An Unexpected Experience of Flow</h3><br />It was a hot day and I was sweaty from walking off the tension of a meeting that hadn't gone well. My thoughts at the beginning of the walk were scattered and disorganized, running through random details of the troubled project I had recently stepped up to manage. The effort to evaluate the project was bringing in more bad news every day. Complex issues and insufficient support for off-shore developers caused compounding delays in delivery. The project was expensive and critical to the future success and value of the organization. Pressure was high. By the end of the walk I remembered that none of these were new or unique problems, and I knew a few possible ways to address each one. I was still new in the position so while pressure overall was high I still had some time to get the project on a better course.<br /><br />Feeling more calm I sat down to review the messages that had come in through the night and check on the status of the few dozen highest priority tasks. My mind jumped all over as I scanned through messages and tasks to get an idea of what was stuck and needed to be clarified and delegated. I felt stressed and scattered as I worked to wrap my head around it all. For so many of the blocked tasks I felt frustrated with no obvious answer of who to delegate them to. With so many of the issues no obvious solution came to mind.<br /><br />Eventually I had scanned through all of them and sat back to consider the most important priorities of the project. I started to feel more relaxed and saw the big picture made up of all these little parts. I suddenly knew which stuck task to address first, just who to delegate it to, and what to write about how to approach the task. That done I jumped to the next task that a solution came to mind for, then to a message where a good answer to the question suddenly became clear.<br /><br />Some time later I looked at the clock on my laptop and was surprised to see that almost two hours had passed. I didn't remember hearing any of the noise in the busy office, even though I was in an open cubicle. I had been interrupted a few times by people stopping by, but even those conversations seemed to fit in fine with what I was working on, and a couple even allowed me to address email messages in person instead of by written reply. I felt content and satisfied, even happy. I was no longer focused and the answers were no longer flowing, but it was okay. My mind felt somewhat empty. I sat for a moment and spaced out, and it felt nice.<br /><br />As I got up to see who was interested in heading out for lunch I realized I had just experienced a deep flow. I was surprised because the conditions for it were so different from the quiet, focused, unpaid free software development work where I usually experience flow. This wasn't development work breaking down details of a single well defined goal, and not even a type of work that I particularly enjoy. It wasn't something I chose to do for its own sake, just something that had to be done. Still, the goal of the effort was clear as was the general purpose that the goal fit into. It wasn't a purpose I cared about deeply for its own sake, it was a paid effort, but the purpose was clear and I was part of making it happen. There wasn't a single task to focus on, and I didn't get to choose the tasks, but I did have flexibility over the order in which to handle the tasks. I also had control over how to handle each one, and at this point had a good enough knowledge of the project and the team so that along with general experience the challenge at hand was fairly level with my ability.<br /><br />Not only did I get through far more in the morning than I had anticipated, and had felt stress about not staying on top of, but my tense and frustrating morning had turned into a pleasant and fulfilling experience. I was full of energy, ready to enjoy a lunch with colleagues, and well prepared to get through the many hours of planned meetings and moving along important objectives that remained in the day. I marveled at the power of flow. I mused over the wide variety of work that could be autotelic, that could be enjoyable and rewarding on its own enough to be intrinsically motivating and not just as part of a bigger external purpose or for extrinsic motivations.<br /><br /><h3>My Free Software Based Career</h3><br />In 2001 I started The Open For Business Project, now Apache OFBiz, a suite of ERP, CRM, and eCommerce applications (<a href="http://ofbiz.apache.org/">http://ofbiz.apache.org</a>). Between then and 2010 I worked around 8,000 hours on it without pay. I then started a series of new free software projects that are a redesign and fresh implementation of a scope similar to OFBiz: Moqui Framework, Mantle Business Artifacts, and various other projects based on them (<a href="http://www.moqui.org/">http://www.moqui.org</a>). I have worked around 2,500 hours without pay on these projects, and anticipate thousands more.<br /><br />Why would any sane person do such a thing? Setting aside the question of sanity, what about the whole issue of earning income for personal needs, family obligations and funding ventures? On top of all that I am a normal person with a wide variety of interests that require time and money such as off-grid living, exploring the wilderness on foot and motorcycle, constant reading and learning, meditation and mindful living, health and fitness, and even a little romance here and there.<br /><br />When I started OFBiz my main motivation was to create something that filled a need I had seen in the two failed startups I had just worked for, and do so in a way that I could build a long-term career on it. Building commercial software does not qualify because if you part ways or the company goes under you can no longer effectively leverage the software you worked on in future contracts or business ventures. It is great to be paid to build something, but even better to build something that you can continue to use and improve no matter where you are working, or if you are working (for money).<br /><br />These free software projects have been a great foundation for a career in enterprise software consulting. I have worked for some of the largest companies in the world, for fascinating and innovative startups, and everything in between. I have experienced business and culture many corners of North and Central America, Europe, and Asia. I have enjoyed a high level of independence and a good amount of work coming in through the free software projects, so I never really had to market myself in other ways. I get to work on the design and development of software that I am good at and enjoy, and not so much on the marketing and sales that I am neither good at nor enjoy very much, and yet still work as an independent consultant or in various incarnations of consulting organizations that I have tried over the years.<br /><br />While these extrinsic motivators are great they are more of a side-effect and what it all comes down to is drive, to intrinsic motivation. This is not just something that drives me to work the long hours necessary for large and complex software projects, but is something that makes the work rewarding for its own sake. It is a powerful thing to experience the flow and satisfaction that comes from the combination of autonomy (of time, task, and team), challenge that often goes just beyond my existing skill and knowledge, and fulfillment in pursuing a greater purpose. These three are key to experiencing flow in creative efforts.<br /><br />Doing creative work in a state of flow is both rewarding (pleasurable and satisfying) and nearly always results in effective, efficient work with excellent results. Working on free software projects benefits from a natural sense of purpose, frequent satisfying challenges, and a high level of autonomy. The experience of flow on free software or other more autonomous creative work carries over into day to day necessary work as well. People learn to get into flow and leverage it for improved results and satisfaction. This is one of the reasons why organizations allowing 20% time to work on whatever employees want see greater innovation and productivity overall.<br /><br />So why do I work on free software? It is a foundation for a career, but more importantly it is something that I can tap frequently for the experience of flow. On the income side of things, it is easier to get into flow doing paid work based on the free software projects that I work on and in the ever growing set of business contexts that I am familiar with. This carries over into paid work and makes it easier for me to get into flow on those efforts. I pursue flow in other experiences like riding a motorcycle in the wilderness, adjusting speed and choosing terrain to challenge and engage myself just beyond my comfort zone (which varies day to day, and even during a ride), but the experience of flow in the context of a larger purpose and creative effort is far more satisfying and rewarding.<br /><br /><h3>Books and Background</h3><br />The best book for an introduction to the topic of intrinsic motivation and flow is:<br /><br />Drive: The Surprising Truth About What Motivates Us by Daniel H. Pink<br /><a href="http://www.amazon.com/Drive-Surprising-Truth-About-Motivates/dp/1594484805">http://www.amazon.com/Drive-Surprising-Truth-About-Motivates/dp/1594484805</a><br /><br />On an interesting side note this book features various stories about Atlassian, the company behind Jira, Confluence, and other software commonly used in projects based on OFBiz, and many software development projects in general these days. Atlassian has a policy of 20% time to work on anything desired, and various other policies that help foster and enable flow. There is some historical overlap between OFBiz and Jira, and Jira uses a customized version the Entity Engine from OFBiz.<br /><br />For those interested in extreme sports and human performance in general, this is a great book:<br /><br />The Rise of Superman: Decoding the Science of Ultimate Human Performance by Steven Kotler<br /><a href="http://www.amazon.com/The-Rise-Superman-Decoding-Performance/dp/1477800832">http://www.amazon.com/The-Rise-Superman-Decoding-Performance/dp/1477800832</a><br /><br />See also the Flow Genome Project (<a href="http://www.flowgenomeproject.co/">http://www.flowgenomeproject.co</a>) which is based on this work, and in particular a great slideshow they put together that describes the concept of flow and trigger for it:<br /><br /><a href="http://www.slideshare.net/StevenKotler/17-flow-triggers">http://www.slideshare.net/StevenKotler/17-flow-triggers</a><br /><br />Much of this is based on the extensive work of Mihaly Csikszentmihalyi. His most business-oriented book is:<br /><br />Good Business: Leadership, Flow, and the Making of Meaning by Mihaly Csikszentmihalyi<br /><a href="http://www.amazon.com/Good-Business-Leadership-Making-Meaning-ebook/dp/B002J05GPS">http://www.amazon.com/Good-Business-Leadership-Making-Meaning-ebook/dp/B002J05GPS</a><br /><br />For those who want to go deeper he has written various other books, and these are a couple of good ones to look into:<br /><br />Flow: The Psychology of Optimal Experience by Mihaly Csikszentmihalyi<br /><a href="http://www.amazon.com/Flow-The-Psychology-Optimal-Experience/dp/0061339202">http://www.amazon.com/Flow-The-Psychology-Optimal-Experience/dp/0061339202</a><br /><br />Creativity: The Psychology of Discovery and Invention by Mihaly Csikszentmihalyi<br /><a href="http://www.amazon.com/Creativity-The-Psychology-Discovery-Invention/dp/0062283251">http://www.amazon.com/Creativity-The-Psychology-Discovery-Invention/dp/0062283251</a><br /><br /><h3>The Experience of Flow</h3><br />Flow is a powerful experience of full engagement and focus. It is a state of consciousness, attention, and awareness focused on a particular activity. The parts of the mind and body that are essential for the activity are fully engaged while the remaining parts of the mind and body are suppressed.<br /><br />There are many aspects of the experience and the resources above have much information about not only what people experience, but also correlated changes in brain chemistry and electrical (wave) state and areas of the brain that activate and (more importantly) deactivate. One aspect is that time perception changes, usually a sense of time slowing down or simply no awareness of the passing of time. This correlates with transient hypofrontality, or more specifically change or suppression of the parts of the brain that monitor time and control your perception of it.<br /><br />In some flow experiences even the parts of your mind that perceive your self will change and the boundary between what is you and what is not you will shift. In some experiences a tool may seem to become part of you so that no conscious effort is required to use the tool, it is like it is part of you. In other experiences you may feel more connected to the environment or to people around you, as if you are part of a greater whole (and this is an important aspect of team flow... yes, people can flow in groups too!).<br /><br />The most critical aspect of flow is that self evaluation, your inner critic, turns off. You experience no value judgment and confidence becomes a non-question... the part of your brain that might doubt is simply turned off. Impulse control goes away along with activation of impulse focus on the task at hand. This feels like a steady stream of impulse and action based on impulse, a literal flow of continual impulse and action.<br /><br />In a creative effort the parts of the brain that are then able to impulse and flow include the parts related to pattern recognition, linking ideas together, generating new ideas, risk taking, and detailed memory of the experience. When needed for the task there may also be increases in awareness, observation, and input acceptance and processing.<br /><br />That is all the performance side of the experience, which will generally result in extrinsic rewards, but what about the intrinsic rewards? This is where the real motivation for the experience kicks in. During and following flow experiences the brain has an increase in norepinepherine, dopamine, serotonin, and endorphins. It is like being on low levels of speed, cocaine, MDMA, and opiates all at once... but totally natural. It is part of your mind's built in reward for your efforts in learning, growing, creating, and unleashing cultivated abilities.<br /><br /><h3>To Flow and Beyond</h3><br />There are steps before and after flow that you should be aware of to understand how flow fits into the overall creative experience (and other flow experiences as well). Below are descriptions of the 4 main phases. Chances are you will recognize these from past creative and other experiences, and understanding them you can start to recognize them in future experiences.<br /><br />1. Struggle: When first beginning an effort you have a lot of mental noise and scattered thoughts jumping among the value, &nbsp;purpose, and details of what you are doing. This is the phase to load relevant information into your mind by reading or conversing, to recall relevant information, and think about the goal you are trying to accomplish and the purpose of the effort. This is a necessary preparation for flow even if it does sometimes seem unpleasant. The inner critic that turns off during flow is active during the struggle phase. The dominant brain wave in this phase is the beta wave, and the most prominent chemicals in your mind are cortisol and norepinepherine. Focus your intentional effort on loading information and setting a clear goal and vision to reach that goal.<br /><br />2. Clarity: In this phase the specific goal and vision for reaching the goal become more clear and you start to get your head around what you are trying to do. This is a key part of getting into flow and is the stage just before flow. If you don't have a clear enough vision of the effort you will never reach this phase and never transition into the flow phase. This phase may be brief or may last a while. To get here you need to relax and move your attention away from your thinking mind to let your mind do its thing. Intentional breathing, physical activity, or a simple distraction can help with this. You will eventually have a clear vision (not necessarily an image) of the goal, and what you need to do first becomes clear. Don't try to think through the whole effort in detail, just get the vision and the first step clear then start acting. Flow is all about engaging in that first step and letting the remaining steps "flow". What is literally happening is various parts of your mind are now doing the tasks you setup in phase 1 and coordinating with your conscious mind as needed by raising impulses for you to act on. Just focus on the first step of your effort, and let the rest come as you go. The prominent brain wave here is the alpha wave, and in this phase nitric oxide in the blood stream starts to increase, which in turn increases blood flow and the delivery of various chemicals to the brain that it needs to function optimally.<br /><br />3. Flow: This is where the magic happens and you experience everything described in the previous section of this article. You start to feel fully conscious (associated with theta brain waves) and at the same time various parts of your mind become active while other parts of the mind that normally suppress these active parts become inactive (the inner critic turns off, time perception changes, etc). Most of the creative work moves to the background of your mind, your subconscious, leaving your conscious mind to act. You will start to efficiently process information and have more frequent and profound insights, which are associated with gamma brain waves. The more you can focus on the effort and let impulse and action flow the longer you will stay in this phase. This can be done with other people. Sometimes things others say and do will become a part of your flow, sometimes they will break your flow.<br /><br />4. Detach and Recover: This phase always happens, and it is best if you allow the time and space for it instead of getting distracted or moving on immediately to something else. You need this phase to recover and learn from the effort, and to prepare for future flow. In this phase you may feel totally spaced out, empty minded. Your mind goes into this state as a natural way to recover from, and process what, happened during the flow phase. When your whole mind is focused on a certain task there may be memories, emotions, sensations, and observations that some part of your mind notices but that don't result in an impulse to your conscious mind because they are not relevant to the effort at hand. Those may come out during this phase. The prominent brain wave in this phase is the delta wave, and both serotonin and oxytocin may be elevated. This can be a very pleasant phase but can also be a let down from the high of flow. Either way it is a valuable phase for learning from your experience and preparing for the next time you do it. For those who meditate this is an excellent time to intentionally be mindful of what is going on in your mind and body. Your whole physiology is primed for mindful meditation in this phase, and doing so will maximize the benefits and the enjoyment that come from it.<br /><br /><h3>Finding Flow</h3><br />That sounds like a great experience, what can I do to get into flow, especially when doing something creative like developing software?<br /><br />Ultimately flow is a mental state, something your mind does to respond to focused, high value effort. You can't force flow, but there is a lot you can do to create the conditions for it, to train yourself to more readily go into and stay in flow, and even change your brain chemistry and electrical state to facilitate it.<br /><br />The Flow Genome Project has a list of 17 flow triggers, and there are three that are most critical to software development and similar creative efforts:<br /><br />1. Sense of Control (autonomy + competence): Autonomy is all about independence and choice in the effort. The three main aspects of autonomy are time (when to do it, how long to spend doing it), task (what to do, how to do it), and team (who to do it with). Different people will consider different of these to be most important, and will be affected in terms of flow differently by each type of autonomy. Competence is the other side of the sense of control. With a higher level of experience and competence it is easier to get to a vision for how to go about the goal, and it is easier to quiet the inner critic that can block flow.<br /><br />2. Strong Purpose and Clear Goal(s): A purpose that you care about will naturally engage you more. The more clear a goal is the more easily you will be able to put together a vision for reaching the goal and identify the first step to get there. The goal and your vision for achieving it is likely to change during an effort, and while it doesn't need to be perfect or even fully complete you need something to start with.<br /><br />3. Challenge/Skills Ratio: For anything you might do you have a range of challenge (or intensity or stress) that you will handle well. There is a low end to this range below which you will have difficulty getting and remaining fully engaged. In other words you'll get bored and more easily distracted, it will be difficult to remain focused. There is a high end above which your skill and experience are not adequate and you will feel overwhelmed and doubtful or even fearful. The sweet spot, where it is most easy to get into flow, is a challenge level that is just a bit higher than your comfortable skill level. Flow happens best when you are fully engaged and this happens most naturally when you push yourself to new levels of performance and achievement.<br /><br />Given these important conditions for getting into flow, how can you train and improve your ability to do so? One of the most powerful practices is meditation. The ability to be fully present in both time and place, focusing on the here and now, is cultivated with meditation and with practicing presence in everything you do. Even planning for the future or pondering the past can be done fully present, just chose the moments and focus on those things. If conditions are ideal for flow then it doesn't matter so much as your mind will shift into full presence, but it is much easier to learn to be present than to always try to setup perfect conditions or be limited to the perfect conditions. Being present reduces the impact of lack of control (both lack of autonomy and lack of competence), makes it easier to accept and embrace purposes and goals, quiets your internal critic, and widens the range of challenge that you will handle well.<br /><br />When a challenge is well beneath your ability level increasing your attention will keep you engaged and present even though the intensity of the challenge doesn't push you there. When a challenge is beyond your comfortable ability being present will keep your mind from running to all the possible horrible outcomes, or the failures of the past. This is the essence of equanimity which is cultivated through meditation, various other mindfulness practices, and practicing presence and attention in everything you do. The ultimate experience of meditation is to detach your consciousness from your thinking mind, body, and emotions so that they are all in the background and your conscious mind is aware of them but not controlled by, or attached to, them. Creative flow is like a meditative experience in that you are more conscious and present in both, and being familiar with conscious awareness will help you get into flow.<br /><br />One good book to get you started, if you are not already familiar with meditation, is:<br /><br />Mindfulness Meditation in Everyday Life and Exercises &amp; Meditations by Jon Kabat-Zinn<br />http://www.amazon.com/Mindfulness-Meditation-Everyday-Exercises-Meditations/dp/149151891X<br /><br />Another tool that appears to have great potential, but that I have not personally tried, is neurofeedback training such as Brain Sport by SenseLabs (was Neurotopia).<br /><br />Okay, that's great as an ideal, but what do I do to function effectively now while I'm practicing meditation and getting into flow? Be careful not to force flow, the extra stress will actually block it. Still, there is quite a bit you can do to "hack" flow. For example if a challenge is too easy you might intentionally make it more difficult, closer to the edge of your skill level, by doing it more elaborately, constraining it in some way (limit on time, size of result, etc), or even doing something passive at the same time such as listening to music or doing something physical (walking, stretching, etc). If a challenge is well beyond your skill level you can get help, train or study before you take it on, or identify the critical success factors and see if you can simplify the effort to something closer to the comfortable edge of your skill level.<br /><br />A lack of autonomy or feeling of competence, or a purpose or goal that don't interest you, are a very different sort of challenge. These become exercises in acceptance. Acceptance is not about going along with things that are harmful to you or others, those are cases where you need to decide to be involved or not and what to do to set and enforce boundaries. Acceptance is about coming to terms with reality, letting reality be what it is without getting upset about it, and even without getting excited or happy about it. Just accept it for what it is, notice it in detail without any value judgment.<br /><br />Once you understand a situation, internal or external, and are okay emotionally with what it is, then you can decide what to do with it and examine your feelings about it. By accepting a lack of autonomy and looking at the situation for what it is, you may find desire and fulfillment in the effort, and be able to flow fully with it even though it isn't ideal. By accepting a purpose or goal that doesn't interest you or even that seems undesirable at first, you may be able to see it and understand all aspects of it more clearly and start to feel interested and excited about it. It is natural when accepting something to be able to notice it more clearly, and natural when noticing something clearly to appreciate it for what it is and even feel gratitude and desire about it.<br /><br />What if I want to get a boost while I'm doing inner work to be able to more fully and frequently flow? One way to suppress some of the same parts of your mind that are suppressed during flow is to use a tDCS device like the "foc.us gamer". This is a useful tool for concentration and can help with focus and turning off the inner critic, which can make it easier to get through the struggle phase into the clarity phase, which is often the most difficult part of getting into flow.<br /><br />Another physical tool to help get into flow is nitric oxide (NO) boosting supplements. Nitric oxide naturally increases in the clarity phase and helps transition to the flow phase by increasing blood flow and delivering more of certain chemicals in the brain. There is a lot of research about nitric oxide and its role in mental and physical performance. There are common parts of fitness supplements that act as NO boosters such as L-Arginine and L-Citrulline. There are also more natural things that increase NO, including green leafy edibles, beets, garlic, cayenne, and good old fashioned exercise.<br /><br />In general anything you can do to get the goal and details about it into your head, produce a vision for the effort, and focus intensely on the effort will help you get into flow. If you know a way to quiet the chatter in your mind effectively and safely then try that to help you get into flow. If you know ways to tune the intensity of a challenge to your skill and full engagement level, give it a try. Improving your ability to flow and the conditions for flow may involve working with others. These may be family, friends, or coworkers. Consider it an opportunity to improve your relationship with them, and maybe even get them interested in flow, presence, acceptance, and other things that will benefit them personally and professionally.<br /><br /><h3>Breaking Down Goals for Ideal Flow</h3><br />Software development involves lots of breaking down bigger goals into specific instructions and algorithm steps that a machine can execute. In a way it is ideal for getting into flow. Once you understand the goal and a general strategy for getting a machine to do that using whatever tools you have available you can get into the procedural flow of laying down one step at a time. Over time your mind gets so used to processes and rules that once you have the general idea of what needs to be done it is almost automatic for your mind to start breaking it down into structural and procedural chunks. Your mind trains well to send you a steady stream of impulses for you to act on as you tap out code.<br /><br />For many of us this mental effort becomes so automatic that we can do it in spite of high pressure and distractions. We can also recover quickly and get back into flow after running into dilemmas and having to step back and observe the problem as a whole for a bit. In fact, we even get good at hopping between different levels of detail to work with higher and lower level processes and data structures and such, and getting into flow at varying levels of abstraction. There is a reason that once we develop some sound skill that we often thoroughly enjoy creating software.<br /><br />What about higher level things, like gathering business requirements and designing user interfaces and other software development tasks where we don't have a nice clearly defined goal to work toward? What about when the task is to clearly define the actual implementation goals like screen designs, data structures, or business rules?<br /><br />This is where levels of abstraction come into play. Business requirements are in effect just a high level, low detail abstraction of the software to eventually create, and preferably written in business terms as opposed to technical terms. Business requirements still live in the context of even higher level business objectives and rules. If you are using effective means of structuring and documenting requirements (such as those in HEMP, <a href="http://www.dejc.com/HEMP">http://www.dejc.com/HEMP</a>) then it will become a natural level of abstraction that you work with just as effectively and efficiently as code. It is a different sort of effort from coding as it is (or generally should be) a collaborative effort with business representatives. The potential when gathering requirements is to frame and direct the conversation in such a way that the whole group focuses on particular business objectives and processes and the whole group gets into flow together laying out the details of the business process.<br /><br />Flow can be experienced in a wide variety of efforts. Even writing this article involved a few distinct periods of flow! Any large project or effort can be broken down into smaller efforts, some of which are efforts to define and break down the larger efforts. Over time breaking down larger efforts to optimize the opportunity for flow with each effort will be a natural part of your work planning and one of your flow facilitating skills.<br /><br /><h3>Getting Started</h3><br />Now you know what flow is and what to look for as you experience different phases of efforts leading up to flow, engaging in flow, and the detaching and recovering from flow. The best way to get started is to simply be more aware, start looking for the phases, sensations, and feelings of flow. Chances are you have experienced flow hundreds of times in your life, and maybe a few times as profound, sustained experiences that you recall clearly. Take a moment to sit in those memories and recall what it felt like, how you got there, similar experiences where it didn't work and what was different that got you fully engaged, present, and focused. In addition to experiencing flow in your work life and creative pursuits you can also experience it in personal efforts like pushing your limits in fitness or physical expression. Paying close attention to what you are doing and the sensations you feel as you do so can be a profound and euphoric experience as you get into flow, and as you get out of it.<br /><br />You have also most likely had experiences of flow with other people, in personal or professional efforts. Those are special and powerful experiences, and teams that do this on a regular basis do amazing things and get profound satisfaction and fulfillment from it along the way. This is one thing people really enjoy about performing music, dancing, and other activities that involve people synchronizing and coordinating with constant engagement and feedback. In a business context this happens a lot during process-driven requirements gathering where people are focused on the overall business objective and work together to layout the steps to achieve it. It also happens during technical problem solving.<br /><br />You can flow in a wide variety of efforts and circumstances, and some lend themselves to flow more than others. Practicing and enjoying flow in efforts that lend themselves to it will make it easier to get into flow in efforts where the conditions are less than ideal. These are often important things to do, critical for business operations and even certain parts of creative efforts and collaboration, and you can do them better and enjoy them more in flow. The experience may be one day a week where you work on whatever you want, in whatever way you want, for your employer. It may be unpaid contribution to free software projects like I have chosen, but it doesn't have to be, and it doesn't even have to be unpaid. The key is to set up the effort to use the triggers and follow the process for getting into flow.<br /><br />Remember that going into flow states for important efforts is a natural part of the function and structure of your mind and body. Important is whatever you decide it is, or more accurately, whatever you tell your mind is important. You get to decide. Flow is the ideal way to learn and grow, to create and express, and your mind and body will reward you well for it.<br /><div><br /></div>David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com5tag:blogger.com,1999:blog-5816842205727721694.post-17250302416867440072014-01-08T03:47:00.002-08:002014-01-08T03:47:54.299-08:00Moqui Ecosystem Application Boost and MentorshipOne of the big objectives right now is to build out the application ecosystem based on Moqui Framework and Mantle Business Artifacts.<br /><br />To help move this along I'd like to start building more community around existing projects (HiveMind and PopCommerce), and support developers working on these and other projects. The general idea is you work on functionality and I will review your work, offer feedback and training along the way, and if applicable (for HiveMind, PopCommerce, Mantle, etc) work with you to get your changes committed to the GitHub repositories for these projects.<br /><br />For HiveMind I've been keeping a list of ideas for additional functionality here, and for those interested this is one good place to start:<br /><br />https://github.com/jonesde/HiveMind/blob/master/WikiSpace/DEMO.cwiki<br /><br />For PopCommerce I have been working on some general ecommerce functionality and there is a lot of work to do based on what is now in place for payment processor integrations (Authorize.net, Cybersource, PayPal, BitPay, etc), shippers (UPS, FedEx, etc), and tax calculation (Vertex, Avalara, etc). My plan is for each of these integrations to be a separate add-on component, BTW, and not part of Mantle or PopCommerce (but they would implement the now existing service interfaces and used the configuration touch points in Mantle).<br /><br />There are also general things like a more real-world friendly checkout process, cart/order based promotions (price rules are already in place), support for products with variants, better browsing and parameterized search help (using augmented ElasticSearch/Lucene search strings like in HiveMind task search), and so much more.<br /><br />These are just some ideas and I'm open to others as well, for HiveMind and PopCommerce and for other projects such as the EZBiz concept which may be split into multiple applications, and should probably have a different name too.<br /><br />If you are interested in participating please send me an email (dej@dejc.com) describing what you'd like to work on and the relevant experience and skills you have.David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com0tag:blogger.com,1999:blog-5816842205727721694.post-22126742671555587182013-12-27T22:40:00.001-08:002013-12-27T23:02:17.382-08:00Moqui Mini-Conf Jan 2014 Las Vegas<div style="border: 0px; font-family: sans-serif; font-size: 13px; line-height: 18px; outline: 0px; padding: 0px 10px 1em; vertical-align: baseline; word-wrap: break-word;"><span style="background-color: black; color: white;">I now have some space in Las Vegas sufficient to host around 10 people for a conference, so planning to do some mini conference and training meetings here more regularly. The place is about 3 miles west of the LAS airport near highway 215. It has a good sized conference table and a connected room with projector setup that is a sort of lounge. </span><br /><span style="background-color: black; color: white;"><br /></span><span style="background-color: black; color: white;">The first of these will be in late January 2014. I am open to feedback on dates and thinking of the 23-25 (Thu-Sat). The first day I'll offer general framework training and diving into more details based on questions and feedback during the training. The training can go into Mantle and HiveMind based on interest, but will be focused on Moqui Framework. The other two days will be general discussions and dev sessions in an un-conference format, in other words we'll start the conference with a huddle about what the rest of the conference will look like.</span><br /><span style="background-color: black; color: white;"><br /></span><span style="background-color: black; color: white;">Some ideas for conference sessions and dev playing include the Mantle data model and services, playing with Drools rules (now used in Mantle for pricing, order shipping charges, and working on the tax calc side of it now) and jBPM workflows. The HiveMind project management and wiki functionality, ideas for improving it, and discussion about real-world use is another good topic right now. </span><br /><span style="background-color: black; color: white;"><br /></span><span style="background-color: black; color: white;">Other fun topics might include ElasticSearch and a follow-up on the results of some ideas that came out of the last conference (the DataDocument, DataFeed, and DataSearch tools, and the Notification framework). For the notification framework a good topic of discussion would be using that with WebSocket, which I haven't done yet due to trouble finding a good library for WebSocket that is independent of the servlet container.</span><br /><span style="background-color: black; color: white;"><br /></span><span style="background-color: black; color: white;">Given the legacy of Moqui/Mantle/etc in the Apache OFBiz project discussions around the two projects (differences, similarities, ideas for using them together, ideas for changing each to be more like the other, etc, etc) is another good topic for discussion. </span><br /><span style="background-color: black; color: white;"><br /></span><span style="background-color: black; color: white;">The last conference was free, and the thanks goes to Hotwax Media for hosting the event. This conference will have a small charge of $50 for basic supplies and snacks and such, and $200 for the training day on Thursday for those interested. </span><br /><span style="background-color: black; color: white;"><br /></span><span style="background-color: black; color: white;">Please comment on the Moqui Ecosystem LinkedIn group on general timing and interest, and contact me directly (dej@dejc.com) if you would like to attend.</span></div><br />David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com0tag:blogger.com,1999:blog-5816842205727721694.post-1293114629410034392013-11-27T09:52:00.002-08:002013-11-27T09:52:33.766-08:00Initial Releases of Mantle Business Artifacts and HiveMind Project Manager<b>Mantle Business Artifacts 0.5.0</b> is now available for download through GitHub here:<br /><br />https://github.com/jonesde/mantle/releases/tag/release-0.5.0<br /><br /><b>HiveMind PM (Project Manager) 1.0.0</b> is now available here:<br /><br />https://github.com/jonesde/HiveMind/releases/tag/release-1.0.0<br /><br />It is exciting to release these first versions of these two projects. They represent the first significant body of business functionality based on Moqui Framework and make Moqui and Mantle an excellent alternative to Apache OFBiz and other open source ERP, CRM, and eCommerce open source projects and commercial products.<br /><br /><h3>Mantle Business Artifacts 0.5.0</h3>This is the initial release of Mantle Business Artifacts (version 0.5.0). It is based on Moqui Framework 1.3.2. This version covers procure to pay, order to cash, and work plan to cash processes including:<br /><br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Purchase and Sales Orders (for goods, services, materials, etc; POs for inventory and equipment/supplies/etc)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Project, Task, and Request management with time and expense recording, billable/payable rates by project/task/client/worker/etc<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Incoming and Outgoing Invoices with a wide variety of item types and an XSL:FO template for print or email<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Automatic invoice generation for purchase orders (AP), sales orders (AR), project client time and expenses (AR), project vendor/worker time and expenses (AP)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Payments, both manually recorded and automatic through payment processing interfaces; applying payments to invoices<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Fulfillment of sales orders (including basic picking and packing) and receiving of purchase orders<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Inventory management including issuance and receipt, and inventory reservation for sales orders<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Automated GL account posting of incoming and outgoing invoices, outgoing and incoming payments, payment application, and inventory receipt and issuance<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>General GL functionality for time periods, validation of transactions to post, time period closing<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Balance Sheet and Income Statement reports (and basic posted amounts and account balance by time period summaries)<br /><br />The automated tests are a great example of functionality and a good reference for where data ends up. They include complete flows with service calls and data validation for purchase orders (procure to pay), sales orders (order to cash), and billable projects with client and vendor/worker invoice/payment (work plan to cash).<br /><h3>HiveMind PM 1.0.0</h3>HiveMind PM version 1.0.0 is the first official release. It is based on Moqui Framework 1.3.2 and Mantle Business Artifacts 0.5.0. This initial version of HiveMind Project Manager includes:<br /><br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Vendor, Internal Org, Client, Worker, and other User administration<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Rate setup for client billing and vendor/worker payout by project, worker, etc.<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Project management<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Milestone management and planning<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Task management (hierarchical, associated with project and optionally milestone)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Flexible task searching and all-in-one task summary<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Request management, including creating tasks to address the request<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Time entry by task<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Expense entry by project<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Invoices and payments to pay vendor/worker for time and expenses<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Invoices and payments to bill clients and receive payment<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Invoice PDF<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>GL posting for invoices and payments<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Balance Sheet, Income Statement, and posted balance/summary reports<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Wiki with attachments for requirements and designs, linked to tasks and requests<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Consolidated searching across projects, tasks, requests, and wiki<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>•<span class="Apple-tab-span" style="white-space: pre;"> </span>Simple email notifications for task, wiki, etc updates<br /><br />The HiveMind-1.0.0-src.zip and HiveMind-1.0.0.zip include Moqui Framework and Mantle Business Artifacts and are ready to go. The HiveMind-1.0.0.zip file is built and has a Derby database preloaded with demo data, so just run it (gradle run, ant run, or java -jar moqui-1.3.2.war) and in your browser go to:<br /><br />http://localhost:8080/apps/hm/<br /><br />David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com4tag:blogger.com,1999:blog-5816842205727721694.post-60977049895813632082013-11-27T09:42:00.002-08:002013-11-27T09:42:26.772-08:00Latest Moqui Framework Release<h3>Recent Releases Summary</h3>It has been a while since I post about the latest in Moqui Framework, so here is a summary of major things in the last few releases (1.2.0, 1.3.0, 1.3.1, 1.3.2):<br /><br /><b>Moqui Framework 1.2.0</b> is a minor new feature release and a major quality improvement release. This release has undergone significantly more testing than previous releases because of a wider variety of functionality that has now been built and tested using the framework, and because of a unit testing effort (framework unit tests built using Spock).<br /><br />The new features include popup menus (using the new jQueryUI menu widget), dynamic-options in XML Form drop-downs with dependency on other fields, automatic optimization of queries on view-entities to old select member entities necessary, support for UUID generated primary keys, expanded JCR support and an Example Content screen, and a number of small improvements that generally make the framework easier to use and more reliable.<br /><br /><b>Moqui Framework 1.3.0</b> is a major new feature and bug fix release. The major new features include the EntityFacade Data Document, Data Feed, and Data Search (based on ElasticSearch) features, the User Notification Message feature, and various improvements to XML Screens and Forms. This release is backward compatible with Moqui Framework release 1.2.0 with the exception that the StatusValidChange entity is deprecated by the new StatusFlowTransition entity.<br /><div><br /></div><div><div><b>Moqui Framework 1.3.2</b> is a minor new feature and bug fix release. The main new feature is a write-through per-transaction cache that effectively simulates the database for the scope of the transaction and does all DB updates with a single connection just before commit. This transaction cache is enabled on service definitions using the cache and force-cache options for the service.@transaction attribute. With this in place various Mantle services now run 2-3 times as fast with results validated by extensive automated tests.</div><div><br /></div><div>Another set of new features covers internal profiling for artifact execution (entities, services, screens, transitions, etc). There are various forms of output available including a full artifact call tree with times, a consolidated tree to see own and total for artifacts in context, and a hot spot list by own or total time. This is similar to Java-level profiling tools but focuses on business level artifacts and with a low overhead gathers data always but only generates reports when needed.</div><div><br /></div><div>OrientDB is now included in the framework by default instead of only in an addon component as before. This is useful as a general graph and document database and as an alternative through the Entity Facade for high-write entities like ArtifactHit.</div></div><h3>Moqui Progress Commentary</h3><div>While there are quite a few new features and bug fixes in Moqui Framework over the last few months, most of the work in the Moqui Ecosystem has gone into Mantle Business Artifacts and HiveMind Project Manager which are built on Moqui and are the main large scale projects based on it so far. I'll post separately about Mantle and HiveMind as both were released with a versioned release for the first time a few days ago.</div><div><br /></div><div>The Moqui user community is expanding and the LinkedIn forum is starting to see more traffic. I am also starting to get more frequent calls and emails about new projects using them, and contracting and employment opportunities for projects based on Moqui (and often Mantle, and sometimes HiveMind).</div><div><br /></div><div>Are you interested in getting more involved? The main thing Moqui and Mantle need now are applications built with them. These could be open source or commercial. The application should be something that benefits you, the better it is for you the better it will be for Moqui and Mantle as projects and products grow around them. Moqui and Mantle benefit from increased exposure, but also from feedback, bug reports, feature requests, patches, and so much more than existing efforts based on the projects have significantly helped with already (for around 2 years now).</div><div><br /></div><div>What is coming next for Moqui Framework? This is often on my mind and is mostly driven by what is needed in applications that I am working on (Mantle, HiveMind, and PopCommerce now and then), technologies or features that people ask about and things that are just really cool or have huge potential to improve the flexibility and capability of enterprise automation systems.</div><div><br /></div><div>One of these I have been researching more recently is the huge potential for rule and workflow tools in ERP apps. My interest in these goes back to the very early years of OFBiz, though neither the rule or workflow engines really worked out well and ended up being useful. Today there are extensive and very capable open source rule and workflow packages, and one (RedHat JBoss Drools and jBPM) where rules and workflow work together and share a knowledge base and such.</div><div><br /></div><div>A couple of weeks ago I attended to Building Business Capability conference in Las Vegas and was inspired by how much progress has been made and all the great examples and case studies of applying these tools. This conference covers rules and workflow, and also requirements analysis and such that is another of my favorite topics in the enterprise software world.</div><div><br /></div><div>I don't know when these things will show up in Moqui, but it will only be when there are real-world uses in place for them. Some early features in Mantle that I'd like to base on such tools include price calculation rules and order promotions rules (replacing the simple condition/action tables concept in OFBiz), order and invoice processing and approvals, and a more elaborate integration with HiveMind for managing the manual activities in workflow along with project tasks and such. There is potential for a lot more, but these would be good proof-of-concept sorts of features.</div><div><br /></div>David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com0tag:blogger.com,1999:blog-5816842205727721694.post-41582821114934072932013-07-03T23:10:00.000-07:002013-07-03T23:12:23.434-07:00HEMP Book Now AvailableA few years ago I started working on a refined approach to business analysis and system design to use with Apache OFBiz and other custom enterprise system development projects. This is based on my years of experience doing this work, starting as a developer often frustrated by inadequate and frequently changing designs, and non-existent or ineffective requirements, and later taking on consulting and active management for dozens of such projects.<br /><br />The material I originally wrote was cobbled together and never cleaned up, but now I have written a book on the topic and it is available on Amazon.com:<br /><br /><a href="http://www.amazon.com/dp/148418422X">http://www.amazon.com/dp/148418422X</a><br /><br />Here is a blurb about the book: "HEMP is an agile approach to business analysis and system design for end users, project managers, business analysts, application designers, software architects, application developers, and quality assurance professionals. HEMP complements agile software development methodologies with an equally agile approach for gathering and documenting business requirements and producing system designs to drive application development that will meet the needs of end-user organizations. The practices and artifacts in HEMP are applicable for building new systems as well as customizing existing ones. HEMP is agnostic to programming languages and application frameworks so can be applied regardless of technology preference. This book includes case studies and numerous examples to help you understand the context of recommendations and how to apply them."David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com1tag:blogger.com,1999:blog-5816842205727721694.post-8860627767978943942012-07-22T19:55:00.002-07:002012-07-22T19:55:49.116-07:00Moqui Framework 1.1.0 Now AvailableVersion 1.1.0 includes various bug fixes and library updates. It is also a minor new feature release with added functionality for anonymous authz, enhanced JSON and REST web service support, XML Form extension with database records (DbForm* entities) for all users or particular users, and an Apache Camel integration with Camel running embedded. This version also replaces the only LGPL library (Restone XML-RPC) in Moqui Framework with an Apache licensed library (Apache XML-RPC). This makes Moqui compatible with licensing requirements for projects that are part of the Apache Software Foundation (including Apache OFBiz).David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com1tag:blogger.com,1999:blog-5816842205727721694.post-33644202499216261482011-11-21T09:11:00.001-08:002011-11-21T09:11:36.921-08:00Moqui Framework 1.0.0 Now AvailableThe initial production-ready version of the Moqui Framework was released on 21 Nov 2011. This release includes various fixes and new features to facilitate easy use based on more extensive testing and real-world development using the framework.<br /><br />More examples of different recommended and useful patterns now exist in Moqui Framework itself, and in add-on projects including Mantle, POP Commerce, and others.<br /><br />To download this release, and for Release Notes with more details about this and previous releases, go to:<br /><br />https://sourceforge.net/projects/moqui/filesDavid Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com2tag:blogger.com,1999:blog-5816842205727721694.post-21220886234742905322011-08-03T18:46:00.000-07:002011-08-03T18:47:30.278-07:00Moqui Framework Release 1.0-rc1 Now AvailableThe first 1.0 release candidate of the Moqui Framework (version 1.0-rc1) was released on 3 August 2011.<br /><br />This release includes various small new features based on feedback and actual use, including Apache Shiro for security (authc and authz), build using Gradle which supports Maven repositories and with directory layout changed to follow Maven conventions, the ability to run arbitrary SQL and get the results back in an EntityListIterator, and the Data View tool that allows users to build queries and data exports on the fly.<br /><br />This release candidate also includes improvements based on more testing, including performance testing and profiling to dramatically increase the speed of various operations, especially those done frequently.<br /><br />As a release candidate from this point only minor changes are expected before the 1.0 final production-ready release.<br /><br />Release download files are available on SourceForge:<br /><br />https://sourceforge.net/projects/moqui/files<br /><br />Note that there is a file that also incudes a preview release of the Mantle Universal Data Model running on Moqui Framework.David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com1tag:blogger.com,1999:blog-5816842205727721694.post-40096865418603535492011-07-14T11:07:00.000-07:002011-07-14T11:11:49.803-07:00Moqui Demo Site on Amazon Elastic BeanstalkThere is now a demo site for Moqui Framework at: <br /><br />http://demo.moqui.org<br /><br />This site is running on Amazon Web Service's Elastic Beanstalk service (as a war file in Tomcat 7) and uses the AWS RDS database (MySQL 5.5.12).<br /><br />Deploying this demo site involved a couple of bug fixes for running on Apache Tomcat and on MySQL, and both are working well now.<br /><br />Moqui also now includes some features to make it easier to create a war file with the runtime directory included, even if you just have the war file from the Moqui binary release and the new wartools.xml Ant build file. Just run "ant -f wartools.xml add-runtime" and a new war file will be created from the existing moqui-1.0.war file, your MoquiInit.properties file, and your runtime directory.<br /><br />Enjoy the demo, and enjoy this new and easy way to deploy apps built on Moqui Framework.<br /><br />Also, moqui.org and my business site dejc.com are now also running on Moqui Framework in another AWS Elastic Beanstalk instance.David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com2tag:blogger.com,1999:blog-5816842205727721694.post-17905997299520240622011-04-01T19:40:00.001-07:002011-04-01T19:40:54.591-07:00Moqui Framework Release 1.0-beta1 Now AvailableThe first feature-complete release of the Moqui Framework version 1.0 is now in beta and available for download. This release represents a major milestone for Moqui Framework as it is ready to be used in development of web-based applications ranging from simple sites and applications with a few forms and some content to comprehensive corporate ERP, CRM, and eCommerce systems.<br /><br />For details about what this release includes, and what has been implemented in previous releases, see the Release Notes here:<br /><br />https://github.com/jonesde/moqui/blob/master/ReleaseNotes.txt<br /><br />For general information about Moqui and the Moqui Framework see:<br /><br />http://www.moqui.org/<br /><br />To try Moqui Framework download it through SourceForge, and read the instructions in the RunDeploy.txt file included in the download:<br /><br />http://sourceforge.net/projects/moqui/filesDavid Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com1tag:blogger.com,1999:blog-5816842205727721694.post-48118154818879765942011-03-31T10:35:00.000-07:002011-03-31T10:46:01.629-07:00Why would someone contribute to an open source project?<p>There is an interesting thread in the Open Source group on LinkedIn that started with the message: "why would someone contribute to open source project? what would programmer get in return?" Here is my response to this:</p> <p class="p1">This question is a good one because it goes to the core of how open source software gets created. The fact that there have been so many replies to this question demonstrates its importance (even to those who seem offended by the idea that an individual needs a reason to contribute).</p> <p class="p1">The factors of satisfaction, building skills, and better employment opportunities (for certain types of companies anyway) are, IMO, more symptoms of the underlying reason rather than the reason itself.</p> <p class="p1">What is implied by the question and many of the answers is that intellectual property protection is the only way to earn a living making software. It seems clear to just about everyone the IP law is not good for innovation, which is best recognized by symptoms like satisfaction and greater value in a market.</p> <p class="p1">Along with this IP law is also not in the best interest of software creators, and does not result in greater income for those of us who create software professionally. When we create software for an employer we get a salary and unless we're really lucky the salary is the minimum they can get away with paying for what is often considered a worldwide commodity skill that isn't worth much in the market.</p> <p class="p1">IP law is great for investors, but bad for the software itself and bad for software creators. Open source and free software are the answers to that.</p> <p class="p1">Those who give up on IP protection to make a living will make more money and have more opportunities for satisfying and skill-building work.</p> <p class="p1">I like to consider myself a good example of that, so much so that after a decade of working on what is now Apache OFBiz all of my free time is going into the next generation of free software tools and applications for business use with Moqui and the projects based on it.</p> <p class="p1">In all of this I've made 2-3 times as much over the last decade as I would have by working as an employee, unless perhaps I had hit a C-level position in a larger company by now (which I'm not sure I'm really that interested in anyway).</p><p class="p1"><br /></p>David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com0tag:blogger.com,1999:blog-5816842205727721694.post-89639570641628418492011-03-01T16:40:00.001-08:002011-03-01T16:40:42.445-08:00Moqui Release 1.0-preview2 Now AvailableWith release 1.0-preview2 the Moqui Framework is now nearly complete and has sufficient functionality to build a wide variety of functional and useful enterprise, or less formal, applications. This release includes a functional example application with various screens demonstrating features of the framework and recommended practices and patterns to use when developing with Moqui.<br /><br />While not feature-complete for the planned 1.0 feature set, the 1.0-preview2 release is ready for early adopters. For those interested, this is a great time to try out Moqui for a pet project or even a more formal project that is just getting started and won't be deployed in production for at least a few months.<br /><br />Feedback on functionality and bugs by early adopters would be extremely helpful for the project at this point, both to validate (or invalidate...) designs and to more extensively test the implementation.<br /><br />The release and release notes are available here:<br /><br />http://sourceforge.net/projects/moqui/filesDavid Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com0tag:blogger.com,1999:blog-5816842205727721694.post-34331555534622137662011-02-21T21:50:00.000-08:002011-02-21T22:52:42.076-08:00Free as in FreedomIn his article about free software versus open source Richard Stallman expresses the importance of a focus on freedom:<br /><br />http://www.gnu.org/philosophy/open-source-misses-the-point.html<br /><br />I couldn't agree more. Freedom can be effectively applied as a solution to any modern problem, including software. On the other hand, freedom can also be ineffectively applied or downright misapplied and fail to solve problems in spite of the good goal and direction.<br /><br />Some say Stallman's focus on freedom is dogmatic, other call it communist, and others utopian. I say Stallman doesn't go far enough.<br /><br />When it comes to free software and open source I certainly started out in the open source camp where the utilitarian development was a higher priority than freedom. At the time I simply found the GNU GPL to be overly restrictive, and ironically because of that it is popular among commercial software companies who want a revenue stream from software licensing and also want to market their wares as "open source."<br /><br />In the last decade my understanding of freedom and its importance in the world has changed, and I think I'm starting to understand Stallman's focus on freedom and his approach to achieving it. Understanding isn't agreement though, and in this case I don't agree with the approach at all. It doesn't really represent freedom at all in my opinion, and is in fact a misguided attempt to violently force people into freedom.<br /><br />Violence? Force? Yep. The irony is the use of the "legitimate" force of intellectual property law as a means to promote freedom. The goal of freedom is spot on, but freedom must be chosen and anything that looks like freedom but is brought about through force will always have a hard time rising above mere compliance.<br /><br />From a different perspective, namely that of law makers and enforcers, this idea is pure genius and because of that is promoted frequently and consistently in nearly all modern propaganda, including media and educational institutions. If you want something done, give someone the power to force it to happen. Those who want certain things done get to feel like something is getting done without doing much themselves, and those who want the power get what they want and usually have no liability or accountability to balance their power. The more we try to solve with force, no matter how legitimate it might seem, the more force and centralized power we'll get as a result.<br /><br />Your mom was right: violence is not the answer. If your mom was really smart, she also taught you that government is little more than a monopoly on violence. In some cases, such as countering aggressive force, violence is entirely appropriate. In most cases it is absolutely not.<br /><br />This is where we cross the line from ethics to morals. Ethics are morals within a certain context, and in the context of legitimate use of force through government violence is entirely ethical. In the more absolute context of morals, violence in response to the treatment of ideas (no matter how they are represented) is clearly not justifiable.<br /><br />So which license is really the friend of freedom? Which open source or free software license rises above ethics into the realm of morals and does away with threats of violence? If the license is based on the concept of copyright, it is violent and antithetical to freedom. If the license tries to force people to do or not do certain things in order to comply with copyright law it cannot be a friend of freedom.<br /><br />In general, and certainly with software, there is only one way to not feed the beast of centralized power: don't solicit it's services! If you want free software, with free as in freedom, then the only approach that is moral, and the only approach that will result in more freedom in the world instead of less is to not use copyright law at all. The only way is to disclaim your copy rights. Until there is no intellectual property law in the world, the only way is through no copyright and no license.<br /><br />Let freedom start with you. Disclaim your copy rights. Put your works in the public domain. Don't feed the beast.<br /><br />Not convinced this is a good idea? Consider what various corporations do that you don't like. Do you support the RIAA and the ruin of lives over copying of music? Do you support the big corps and their approach to protecting their "rights" with commercial software? Are you in favor of a $500,000 fine and 5 years in prison if you copy a movie, and the absurd amount of our money through taxes that go to enforce that? Are you in favor of DRM software, and laws against even revealing the details of DRM software? Do you think it's okay to throw people in a cage or steal money from them because they wore a t-shirt with a set of numbers and letters printed on it?<br /><br />I hope your answer to those questions is no. If not, then why on earth are you reading this? So what is it that bothers you about those things? <br /><br />All they are doing is applying copyright law to protect their interests in the "intellectual property" they own. <br /><br />If you do the same, are you any better? Can you claim any moral, or even ethical, superiority? Why should people listen to you?<br /><br />Let freedom start with you. Disclaim your copy rights. Put your works in the public domain. Don't feed the beast.David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com0tag:blogger.com,1999:blog-5816842205727721694.post-80937800118350739472011-02-09T21:40:00.000-08:002011-02-09T21:42:59.137-08:00Moqui Release 1.0-preview1 Now AvailableThe first release of the Moqui Framework is now available for download through SourceForge at:<br /><br /><a href="http://sourceforge.net/projects/moqui/files">http://sourceforge.net/projects/moqui/files</a><br /><br />This first release has many tools that are usable right now, but there are many features planned for the final 1.0 release that are not yet complete. See the release notes for details about what is included and what is not yet either in the distribution or on the download page linked to above.<br /><br />This release includes a WAR file that is executable (like an executable JAR), or can be deployed in an app server or servlet container. For more details see the RunDeploy.txt file in the distribution, or online here:<br /><br /><a href="http://moqui.svn.sourceforge.net/viewvc/moqui/trunk/moqui/RunDeploy.txt">http://moqui.svn.sourceforge.net/viewvc/moqui/trunk/moqui/RunDeploy.txt</a><br /><br />Part of the purpose for this release is to get feedback, bug reports, and contributions. If you have any questions about those, feel free to contact me or post in the forums. Thanks to all for feedback so far, and for continued interest and support.David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com2tag:blogger.com,1999:blog-5816842205727721694.post-80974558395766682612011-01-10T04:12:00.001-08:002011-01-10T04:16:35.489-08:00The Moqui Framework Web SiteI started work on design of the Moqui Framework in Feb 2010. Just a few weeks ago in late 2010 I decided the design was done (I made it through all of my TODOs), and started implementation. Right now implementation is under way, and I'm appreciating the power and flexibility of Groovy and as I get into higher level code also the features of the Moqui API and features.<div><br /></div><div>Anyway, to the point of this post...</div><div><br /></div><div>This weekend I decided it was finally time to get a web site together for the project. This initial web site has useful links and a good deal of content about the project. If you're wondering about what Moqui is or how it will be managed or what drove the design, then there are good time ahead for you at:</div><div><br /></div><div><a href="http://www.moqui.org/">http://www.moqui.org/</a></div><div><br /></div><div>Enjoy!</div>David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com0tag:blogger.com,1999:blog-5816842205727721694.post-15555429041093316622010-02-07T12:51:00.000-08:002010-02-07T13:05:33.869-08:00Why Business Apps Are Difficult, and What to Do About It<span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';">There are lots of questions and misunderstandings about the nature of business applications, and why Apache OFBiz is the way it is, and why it isn't go to hold your hand and guide to just what you want for your business. Sometimes people make comparisons to operating systems or other lower level tools, but there are some key differences that make all the difference and...</span></span><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';"><br /></span></span></div><div><span class="Apple-style-span" style=" ;font-family:monospace;font-size:medium;"><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';">There are a few reasons why the business side of OFBiz (the applications) are a lot more complicated and difficult to learn than the technical side of OFBiz. The basic problem is the size of each, but that's over-simplifying things.</span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';"><br /></span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';"><b>Read Before You Write:</b> It's not really human nature to do this, and it takes a lot of patience. This is made worse because if individuals have a hard time with patience, organizations are simply incapable of it. The PHBs want results... and reading sure doesn't look like it's producing any. What's worse is if the individual manages to produce a result with a couple of dozen lines of configuration and bits of code instead of a couple of thousand lines of raw, meaty, manly Java then a semi-technical PHB may find it really unsatisfying to have paid for so much time to get so little, not realizing that the individual just saved him 10-100 times what the alternative would have cost initially and over its useful lifetime. Unfortunately some individuals DO realize this and build large, profitable businesses based on exploiting it (but that's another topic altogether).</span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';"><br /></span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';"><b>Scratching the Surface:</b> A business application is not like an operating system, or even a framework for building business applications (which is like an operating system, except the interfaces are tuned to a different type of input, a less technical and more business-oriented type of input). The difference is by nature there is no way to design an interface adequate to represent a business application, and that is what both operating systems and application frameworks are all about. Unfortunately business people don't like being told that an interface with a few little parameters is supposed to represent the entirety of options for ANY process in their business, even "standard" ones like billing or shipping. Business people don't like not be able to change and tune any part of their business that they want, and if the systems can't keep up then they don't get used. SAP and most ERPs out there are great examples of this. They are proprietary software works and you don't get all of the source code (and/or permission to change it), and can only change what they've decided it's okay to change (unless you want to rewrite something, usually more than you think). Those sorts of systems don't let you get below the surface, which is unfortunate because then you don't even have an option to Read Before You Write.</span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';"><br /></span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';">In other words, the way you go about adding value (via easy of reuse) in a business application is very different from how you go about adding value in an operating system or a business application framework. With the OFBiz framework you can learn the "interface" to it, but with the applications you pretty much have to deal with it all. On the other hand, there are more concise "interfaces" to it, like the data model and browsing things related to data model elements, which is made easier with the Artifact Info and other related tools in the OFBiz WebTools application.</span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';"><br /></span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';">And how do you apply that in your business? The basic answer is you don't. OFBiz is meant to adapted to businesses, not businesses to OFBiz. You can certainly run it OOTB, but that's not how it's meant to be used and you'll find that a painful experience. It's not going to hold your hand because it was never designed to run your business. Frankly, how could it be? Some systems claim they are in their marketing, but that marketing isn't honest because how do they know how you want to run your business? When you start trying to use those systems in your business you find out pretty quick that they really don't know.</span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';"><br /></span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';">So, your best bet is to define your business and then do a gap/overlap analysis with OFBiz to see what you can use, what needs to be adapted, and what needs to be built to fill gaps. If you really want a tuned system, like for a larger company or for a derivative work (like a commercial application targeted at a certain type of company) then you can define the business, design the application, and build it, and save resources building it by reusing as much as possible from something like OFBiz (which gets back to why OFBiz is organized like it is). To do these things effectively takes some experience, and to shorten the path certain tools are helpful like the HEMP approach (http://www.dejc.com/home/HEMP.html).</span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';"><br /></span></span></div><div><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-family:'times new roman';">Stepping back a little... there is a bigger trick... and that is how many people believe what I wrote above Read Before You Write and Scratching the Surface? Well, not many (that I've had the pleasure to meet anyway). For those that do understand and agree OFBiz is great (could be better, lots better, because even many people involved with OFBiz don't believe or don't understand those two ideas and as the number of contributors increases that painful fact becomes more apparent). For those who don't understand or don't agree, they are destined to a life of making things painful for them and others they work with, whether they attempt to use OFBiz (probably won't last long) or whether they choose a likely painful commercial route filled with reasons to spend more and more money on more and more different software.</span></span></div><div><br /></div></span></div>David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com27tag:blogger.com,1999:blog-5816842205727721694.post-80677869735540192122009-06-23T13:48:00.000-07:002009-06-23T13:51:12.735-07:00<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica"><span class="Apple-style-span" style="font-size: x-large;">Open Source Community Collaboration Best Practices</span></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">WARNING: This is a long post, approaching a dissertation on the topic. I recommend that you glance over the headings, and then go get some tea and cakes, coffee and donuts, or whatever it is you enjoy consuming casually while having your brain tickled.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica"><span class="Apple-style-span" style="font-size: large;">The Best of Practices</span></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">The title to this article falls a little bit short of accurate... I actually only want to talk about a single best practice that seems to consistently determine whether or not an interaction with others in the community will be successful or not. The idea of this give and take is really simple, and goes back to the basic definition of collaboration:</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">1. the action of working with someone to produce or create something</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">2. traitorous cooperation with an enemy</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">Both parts of this definition are interesting, because the practice that leads to success in community collaboration is one that aligns well with #1 and avoids any appearance of #2. On either side of the definition it comes down to who you choose to collaborate with, and that's the most important thing to keep in mind in an open source project: you are collaborating with people and expecting to both give and take as part of your interactions with them. </p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">If you want other people to collaborate with you, first you need to collaborate with them. If you are hoping for some benefit of the collaboration you first need to set the stage for the collaboration and invite others to collaborate by starting first and giving what you have, then invite others to get involved. The important thing to keep in mind is that collaboration implies a two-way street and if you try to make it one way by not trying to collaborate with others, but still expecting them to collaborate with you, then you will most likely be disappointed by either no collaboration at all, or a good-will attempt by someone else to collaborate with you that will fail because no stage was set for the collaboration and they will likely help in a way you don't need.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">Now a quick "anti-pattern" to explore what this implies... you may be wondering: "how can I be sure that other people will collaborate and I'm not just giving stuff away and will never get anything back?" That issue is certainly one that stands in the way of collaboration and basically represents a lack of confidence in the possibility of collaboration. The problem is that if you are unwilling to give it a chance and "cast your bread upon the waters" as it were, then there is not even a chance that you will be the fortunate recipient of collaboration in return. </p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">If you can't give for the sake of fostering collaboration and good will, then community collaboration may not be for you. The best path may be to continue to collaborate through the financial and legal means that are the common operating mode of commercial software and result in a very inefficient form of collaboration (if you can even call it collaboration) that causes many of the problems that people and organizations face with the software they currently use. Unfortunately many commercial software organizations would consider open source collaboration to be firmly in the camp of definition #2 above. </p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">One good concept to keep in mind when working with others is the "burden of proof." A good way to be generous with others and to make it clear to others that you are putting efforts into collaboration is to take upon yourself the burden of proof, and not ask others to take it on. When you do this you will communicate more clearly and others will tend to reciprocate and do their own research, puts their thoughts and effort into it, and make things flow back and forth more smoothly. What this means in daily practice is usually to just research things well and before proposing something or asking a question thoroughly present your research, including references and links where possible. In the Apache OFBiz Committer's Best Practices Guide this is referred to as "read before you write" and the burden of proof concept takes it one step further to sharing with others what you read so that you can collaborate on the writing side of things.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica"><span class="Apple-style-span" style="font-size: large;">A Case for Collaboration</span></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">This reminds me of a quote from a movie scene (whose name I can't remember and google isn't helping): "Qui bono?" The line is from an old politician who has a clear set of priorities. I'm not referring to the "Qui bono" line from The Departed, though that one is more entertaining, and I guess almost equally as enlightening. Whatever the case, the point is: who benefits? I suppose the point from The Departed is also appropriate: who cares?</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">Commercial software organizations operate under a specific model and that is very understandable from their perspective with their investors and a wide variety of stakeholders that they need to make a profit and leverage whatever assets they have available in order to do so. Software is often considered such an asset, and treating it that way is extremely well supported by modern intellectual property laws. So, is that a good model or a bad model? In the abstract I think that's impossible to answer, and it depends on whose perspective you are looking at.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">What about the perspective of the end-user? Which approach best benefits the end-user (individual or organization)? Is there a conflict between what benefits software producers and consumers? My opinion on this topic is that again the answers depend on who you ask and that nearly all large commercial software producers will answer in their favor, and some end-users may as well (perhaps because they are influenced by marketing messages) though for the most part they will answer differently based on actual experience with those large commercial software producers.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">All of that introduces a lot of complexity, and it is possible to look at the simple flow of information under the assertion that the more end-users have influence over software the better it will meet their needs. Through open source collaboration end-users can very directly influence the direction of the software by participating in the project and collaborating with other contributors and users. And what about the crazy things that some companies do? The collaboration tends to be very effective for filtering out or refining ideas that aren't so great. On the other hand, sometimes the "crazy" ideas turn out to be a differentiator for the company and result in a huge benefit over competitors. With open source software they have the flexibility to implement such things anyway (and do so efficiently) and even keep those changes private if they desire (well, unless the software is GPL licensed, which I would assert is not a good license for business software that is meant to be customized).</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">On a more day-to-day note direct collaboration is powerful in many situations that users and developers run into. It allows new users to get feedback on what they are trying to do and either open the way for new functionality or get recommendations on how to better use existing functionality. On the other end of the spectrum, it allows experienced users and contributors to get feedback on both their requirements, designs and implementation approach and ultimately results in far better software that will generally meet current needs better and also be more likely to handle changing future needs. In other words, whether you are experienced or not direct community collaboration can very efficiently help you to resolve issues that you face, and exploit opportunities that arise.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">In short the ultimate reason for direct collaboration, as opposed to collaboration through financial and legal means, is that it results in software that better meets the needs of end-users (more flexible), better protects the interests of end-users (more private), and is also less costly for end-users. That reason will always be true. </p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">Right now there is another reason to consider more direct collaboration and that is that commercial software companies have become massive entities with a lot of influence over governments as well as end-user organizations... and this means that they are able to put their own interests above those of end-users and get their way as much as they want. Very large end-user organizations can sometimes stand up to them, but that is still a small minority, even in this modern world of business. Shifting that balance to something more in favour of end-user organizations will make non-software commerce more efficient and allow software to be more an enabling industry instead of a controlling one and ultimately enable general economic growth and stability. </p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">For commercial software vendors there doesn't have to be a conflict with open source collaboration. They don't HAVE to consider that collaboration to fall under definition #2 above. It may require some adjustments, and ultimately by better serving their end-users they will offer better solutions and keep clients and customers for the long term. The downside is the sacrifice of short-term profits and the ability to boost profits by pushing clients to buy things at certain points in time, like scheduling releases and upgrades to boost revenue or introduce separate products instead of adding features to existing products and including them in free service packs. End-users don't like such practices and if they perceive a viable alternative they will leave... so why not change practices and find more profitable approaches that also better serve the end-user who ultimately pays all of our bills...</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica"><span class="Apple-style-span" style="font-size: large;">Collaboration on Software</span></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">Going beyond general collaboration there are certain things that must be done to more effectively collaborate when working specifically on software.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">The most important is to collaborate on designs and not just on implementation, keeping in mind that to collaborate you need to start out by collaborating with others and giving them a chance to collaborate with you. In the software world that means don't just implement something and throw it out for everyone to try to digest and react to. Instead, write up what you want to do and give other people a chance to send their feedback. Depending on the complexity of what you are working on this could take from days to weeks.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">So why do this? Won't it just delay your work? Consider that if you create a design on your own chances are you'll forget things or not be aware of requirements that are common and important to that design. If you leave those out and implement something then sooner or later what you implement will be changed, or thrown out and re-written. Why is that? Because important requirements were left out of the initial design, and those requirements won't just go away. If you're lucky the requirements will have a small impact and won't require significant changes to the design or the implementation. However, especially if you are not trying to make things as broadly applicable as possible then it is easy to ignore big things that will impact the design a lot, and then it may be impossible to just refactor and extend the implementation to support the new design.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">How can we avoid this? The answer, as would be expected in an article about collaboration... is to collaborate in advance on requirements and get the design to a solid point before you start implementation. Your code will have a longer lifespan and almost always also be more useful to you and your clients.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica"><span class="Apple-style-span" style="font-size: large;">Some Short How-To Scenarios</span></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">1. I want big feature XYZ to be developed and I would like to work with others on it to ensure the best possible design and so I don't have to implement all of it.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">2. I want big feature ABC to be developed and I'm not a developer so I can't help with development.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">3. I have a question and I'd like to get an answer through the community (I can't afford to pay someone to support me, or I think this is an issue that the community should address).</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">4. There is something that exists in the software that I think should work differently.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">For all of these the pattern is the same. Start with research on what exists in OFBiz, the requirements driving what you want, what can already be done and what can't be done or that you can't figure out how to do, then present your findings and requests for help and collaboration that you would like to see happen.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica"><span class="Apple-style-span" style="font-size: large;">Collaboration Anti-Patterns</span></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">In the software world one way to clearly not collaborate is to totally rewrite something instead of improving something that exists, especially if this involves no attempt to understand what already exists and make sure that the replacement is sufficiently better to be worth the change and also does everything that the old stuff did. To understand more of why this is bad for collaboration, and sometimes bad for design and implementation as well, consider some of the things that you would be implicitly doing:</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Courier"><span class="Apple-style-span" style="font-family:arial;">1. throwing away the feedback (or potential feedback) from others on the mailing lists</span></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Courier"><span class="Apple-style-span" style="font-family:arial;">2. throwing away the various fixes and improvements that people have put in</span></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Courier"><span class="Apple-style-span" style="font-family:arial;">3. in doing #1 and #2 you are making it impossible for people to collaborate with you... as much as many people are trying when you take this approach you are not allowing people to do so</span></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Courier"><span class="Apple-style-span" style="font-family:arial;">4. large scale re-writes when there are issues instead of addressing the issues usually results in things NEVER becoming refined (since the focus is on starting over instead of on refinement), and in addition leaves a very large wake of orphaned code</span></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica"><span class="Apple-style-span" style="font-size: large;">Final Summary</span></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">In short the best way to collaborate with others, especially in a volunteer situation like an open source project, is to start by investing in the direction you want and then giving that investment away and soliciting involvement from others (including a write up on what you've done and your vision of what it can/should be and perhaps even on how others might get involved). By "casting your bread upon the waters" in this way you open the way for others to join in and give back, to collaborate back with you. Others may not collaborate back in the way or on the timeline that you had in mind, so be patient on both aspects. Consider what others suggest and appreciate their suggestions and discuss them in the open. This will foster more collaboration and allow the end result to be far superior to what you are capable of on your own... no matter how talented and/or experienced you are.</p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br /></p> <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">This direct collaboration model is a powerful way of getting things done and maintaining them over long periods of time in inherently stable ways. It enables a high degree of innovation and eventually results in solutions that are eventually clean and efficient and effective. Again, be patient with it as things are generally not perfect on the first pass or during the height of the collaboration activity, but rather when the collaboration has taken its course.</p><div><span class="Apple-style-span" style="font-family:Helvetica, fantasy;font-size:100%;"><span class="Apple-style-span" style="font-size: 12px;"><br /></span></span></div>David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com12tag:blogger.com,1999:blog-5816842205727721694.post-68407778718788721782009-06-10T02:08:00.000-07:002009-06-10T02:10:45.306-07:00A Time for Change<span class="Apple-style-span" style="font-family: 'Lucida Grande'; font-size: 11px; white-space: pre-wrap; ">I have had the pleasure of working with a number of great people over the 8 years I've been involved with OFBiz. For the last 2.5 years I've been a part of a great and growing company, Hotwax Media, and it has been amazing to see the number of people involved with Apache OFBiz increase as well as the work done over the years (including excellent custom work and a large number of contributions back to the open source project).</span><div><span class="Apple-style-span" style="font-family:'Lucida Grande', fantasy;font-size:100%;"><span class="Apple-style-span" style="font-size: 11px; white-space: pre-wrap;"><br /></span></span></div><div><span class="Apple-style-span" style="font-family: 'Lucida Grande'; font-size: 11px; white-space: pre-wrap; ">Much of the world seems to be changing right now and more changes are coming as people adjust their priorities and move on to new opportunities. For me personally the time has certainly come to make some changes. I'm cutting back on both expenses and investments, and getting back to what I know best and enjoy doing most: independent consulting. </span><div><span class="Apple-style-span" style="font-family:'Lucida Grande', fantasy;font-size:100%;"><span class="Apple-style-span" style="font-size: 11px; white-space: pre-wrap;"><br /></span></span></div><div><span class="Apple-style-span" style="font-family: 'Lucida Grande'; font-size: 11px; white-space: pre-wrap; ">On Saturday June 6th I resigned from Hotwax Media, leaving behind a great group of people and even some cash on the table, and moving toward the many and growing opportunities for doing other OFBiz-related work. Apache OFBiz is a great solution in an economic environment (similar to the one it started in) where organizations are moving back to core values and focusing on accomplishing their core objectives. In some cases that means doing more with less, and in nearly all cases it means doing more of what distinguishes the organization, and doing that requires more control over the business and the software that helps automate and manage it.</span><div><span class="Apple-style-span" style="font-family:'Lucida Grande', fantasy;font-size:100%;"><span class="Apple-style-span" style="font-size: 11px; white-space: pre-wrap;"><br /></span></span></div><div><span class="Apple-style-span" style="font-family: 'Lucida Grande'; font-size: 11px; white-space: pre-wrap; ">While it's been great working with the people at Hotwax and with many clients through the company, I miss working more with others in the community and seeing more of what other service providers and end-users are doing. I'm truly looking forward to doing work similar to what I did a few years ago and working with a larger group of individuals and organizations.</span><div><div><span class="Apple-style-span" style="font-family:'Lucida Grande', fantasy;font-size:100%;"><span class="Apple-style-span" style="font-size: 11px; white-space: pre-wrap;"><br /></span></span></div><div><span class="Apple-style-span" style="font-family: 'Lucida Grande'; font-size: 11px; white-space: pre-wrap; ">Along with this I'm also looking forward to putting more time and effort into new objectives for Apache OFBiz, including adding collaboration on requirements and designs to the existing highly successful collaboration on implementation, and along with that creating some applications meant to be used out-of-the-box by specific types of organizations or individual end-users.</span></div><div><span class="Apple-style-span" style="font-family:'Lucida Grande', fantasy;font-size:100%;"><span class="Apple-style-span" style="font-size: 11px; white-space: pre-wrap;"><br /></span></span></div><div><span class="Apple-style-span" style="font-family: 'Lucida Grande'; font-size: 11px; white-space: pre-wrap; ">If you think I might be able to help you with something you have going, please look at my web site at http://www.dejc.com/ for contact information and more details on the types of services I am offering. </span><div><span class="Apple-style-span" style="font-family:'Lucida Grande', fantasy;font-size:100%;"><span class="Apple-style-span" style="font-size: 11px; white-space: pre-wrap;"><br /></span></span></div></div></div></div></div></div>David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com1tag:blogger.com,1999:blog-5816842205727721694.post-75380741703863054952009-05-12T01:18:00.000-07:002009-06-22T18:14:49.744-07:00Apache OFBiz at OSCON 2009<div><br /></div><div>O'Reilly OSCON 2009 is taking place in San Jose, CA, USA on the 20-24 July 2009. For details on the conference see:</div><div><br /></div><div><a href="http://en.oreilly.com/oscon2009">http://en.oreilly.com/oscon2009</a><br /></div><div><br /></div><div>Apache OFBiz was going to have a booth there but that has been cancelled because I was unable to line up enough resources to make it happen.</div><div><br /></div><div>An OFBiz-related thing at the conference that is still happening is the presentation I will be giving on Thursday the 23rd at 2:35 PM. The title of the presentation is "Apache's Open For Business: Holistic Analysis and Design." For more details about the presentation please see the schedule here:</div><div><br /></div><div><a href="http://en.oreilly.com/oscon2009/public/schedule/detail/7453">http://en.oreilly.com/oscon2009/public/schedule/detail/7453</a><br /></div><div><br /></div><div>I look forward to seeing everyone there and enjoying some conversation and collaboration during a bit of the Northern California Summer.</div><div><a href="http://conferences.oreilly.com/oscon" style="text-decoration: none;"><span class="Apple-style-span" style="color:#000000;"><br /></span></a></div><div><a href="http://conferences.oreilly.com/oscon"><br /><img src="http://assets.en.oreilly.com/1/event/27/oscon2009_banner_speaking_210x60.gif" width="210" height="60" border="0" alt="OSCON 2009" title="OSCON 2009" /><br /></a><br /></div>David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com0tag:blogger.com,1999:blog-5816842205727721694.post-81079401308613403042009-04-13T10:38:00.000-07:002009-04-13T10:40:45.113-07:00Apache OFBiz Community Building Tour - Mid-western USA, April 2009As a follow-on to the April release branch, and to help encourage community growth and participation for Apache OFBiz, I'm planning a tour of the Mid-western United States. As part of this tour I'd like to visit with individuals and organizations who are current and prospective users, contributors, and others involved with or interested in OFBiz.<br /><br />In these visits I will be available to spend 1-3 hours with you and answer questions about Apache OFBiz and how you can use it most effectively. I would be happy to speak with individuals or as many people from your organization as you would like.<br /><br />If you would like to better leverage opportunities to collaborate with others in the community or there are specific things you'd like to see in OFBiz, or be able to do with OFBiz, this is a great time to chat about it. Also if you'd like general business level or technical help with the software I'm happy to go over those sorts of things as well.<br /><br />I am doing this as the PMC Chair of Apache Open For Business with the intent of helping grow the community and not in my role as an officer of Hotwax Media. If you are interested in the services of Hotwax I'll be happy to briefly answer questions and refer you to the Hotwax sales people.<br /><br />Below is the rough schedule I have in mind and the general areas I plan to be in. I'll be changing it as needed to accommodate what people are available for. If you're around these areas and available around these times, please let me know!<br /><br />17 April (Fri): Omaha NE, Des Moines IA, Kansas City MO<br />20 April (Mon): St. Louis MO<br />22 April (Wed): Indianapolis IN, Louisville KY (maybe Chicago IL)<br />24 April (Fri): Memphis TN, Nashville TN<br />27 April (Mon): Dallas TX, Oklahoma City OK<br /><br />Please contact me directly by email if you're interested in a visit. If you know of someone else who might be interested in a visit, please make an introduction and if they are interested I'm happy to play along. My schedule is flexible on this trip so morning, lunch and evening visits along with normal business hours are fine.<br /><br />There is no fee for this, but I won't refuse a free meal if offered. I'm traveling in a 40' motorhome, so hints for parking somewhat nearby are also appreciated.<br /><br />I am planning to do this in other areas in the near future, but don't have firm plans yet. The next likely area will be the north-western USA in July (around the time and place of OSCON which I'll be speaking at on July 23rd in San Jose, CA).David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com0tag:blogger.com,1999:blog-5816842205727721694.post-45230350864998021982008-01-31T17:34:00.000-08:002008-01-31T17:47:41.470-08:00Re-Reference Post: Community versus Code and Glass CathedralsSome feeds missed this post (or weren't setup yet), so this is a short post to point back to it. There are some ideas of interest to the open source world, and as a bonus I've made an attempt at coining a new term: Glass Cathedral.<br /><br />http://osofbiz.blogspot.com/2008/01/glass-cathedrals-and-community-versus.htmlDavid Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com0tag:blogger.com,1999:blog-5816842205727721694.post-3855096497918335172008-01-30T21:51:00.000-08:002008-01-31T17:40:15.979-08:00Glass Cathedrals and Community versus Code<div><span class="Apple-style-span" style="font-weight: bold;">Collaboration</span></div><div><br /></div>From the proverbial "Day One" of thinking about OFBiz it was clear to me that I couldn't do it alone, and in the financial environment of the day (early 2001) it was also clear that there was no way I could raise the money necessary to make the dream a reality. That was especially true for me because I had nothing in my background other than a couple of years of research and failed commercial attempts to set me apart and make me of interest to prospective investors and partners.<br /><br />Enterprise systems in general are large, complex things and require hundreds of thousands of man-hours to implement. On top of that with such a wide variety of businesses and real-world requirements designing something that would be flexible enough to be a good starting point and that would not require rewriting significant parts of the software would require a body of knowledge and experience that would be almost impossible to pay for. To start from scratch and build such a system would require a lot of collaboration.<br /><br />The traditional means of collaboration in the commercial software world is to pay for a license and hopefully influence the vendor to add or fix things. There is little influence in this. Some vendors will accept payment for development of specific features, but for the most part customizations and specific features are built outside of the core product and never shared. This is a very limiting form of collaboration.<br /><br />I worked with open source software before getting into all of this enterprise systems stuff as part of a company called Lineo that was creating products for embedded systems based on Linux, and in college as well. While doing R&amp;D for an ecommerce startup I also ran into a number of open source alternatives, but mostly in infrastructure software, there just wasn't anything real happening in enterprise automation at the time.<br /><br />The collaboration problem and the open source solution seemed the perfect match. Collaboration through a non-profit, open source effort seemed like a significantly more efficient way to go about things that collaboration through commercial licensing and the often disposable results of consulting engagements. With no financial entanglements collaboration can focus on solving problems and creating solutions with a long life span. Speaking of life span, the likelihood of survival for a piece of software is much greater when no one company can fall and take the software with it.<br /><br />And there was more. I saw quickly that in addition to making the development of such a system from scratch possible, it also made the results of the development effort far better than any individual or static team working in isolation ever could. I started to see the lack of funding and the necessity of working on contracts to survive as a great benefit to the effort rather than a distraction from it. In short if I had personally had all the time and money I needed to site in a room and build this system, or even just the framework part of the system, it would not be anything like the real world need driven framework and application set that it is today.<br /><br />The collaboration was just amazing. So many people suggested things over time and even though the majority of the suggestion never yielded anything immediately fruitful, there were so many that were critical and that pushed us in the direction we wanted to go by paths we hadn't even considered. I'm comfortable admitting that there is no way I could have built OFBiz the way it is now on my own. Over time I have also learned to accept that in many cases things I built and thought were great were inferior to existing alternatives or ideas about better ways of doing things. In the first 2-3 years of OFBiz we threw out as many lines of code as we kept. There was continual housekeeping and refinement, and there still is today even though these days the patterns and practices are more established and tried and proven, so the need for this is less frequent (on a large scale at least...).<br /><br />After running OFBiz this way for years there were various people and organizations involved and this means of collaboration was proving to be very successful and stable. We were certainly not the only ones to run a software project this way and in 2006 some people from the Apache Software Foundation approached us, recognizing the similarities in philosophy, and inviting us to join the ASF. The more I read about Apache the more I came to see things this way as well, and the more I could see that they had the same emphasis on how to structure an open source project that I had come to. The great thing was that they had a much better organization and much better legal infrastructure (including the Apache License 2.0).<br /><br />One of the main tenets of the Apache Software Foundation is that the community is the most important thing, and if you get that right then good code and other things naturally follow. In other words, if people are brought together in a way that enables minimally encumbered collaboration they will create something good, and something far better than if that collaboration had not been present.<br /><br /><div><span class="Apple-style-span" style="font-weight: bold;">Code</span></div><div><br />This may seem obvious to many people these days. These sorts of concepts have come a long way in recent years, even outside of the once fairly limited open source development circles. So why am I bringing this up now, and writing about all of this nearly 7 years after the inception of OFBiz?<br /><br />The answer is that the principles and efforts of community driven open source projects have as much impact today as ever, and are a key to distinguishing efforts in the so called "open source" world.<br /><br />There are some in the real open source world that don't focus on community and collaboration, and suffer because of it. One big potential problem for open source projects is a forked code base, or more importantly: a forked community. Sometimes a community degenerates or there never really was a strong community at all and a fork is appropriate.<br /><br />In many cases though people fail to recognize the importance and value of the community and collaboration within it. For whatever reason they decide that they can do better on their own (often along with select others). They decide that the code is more important than the community.<br /><br />In the OFBiz world there are various examples this failure to collaborate resulting in projects largely based on OFBiz but that have chosen to go in different directions. Many of these are open source projects, but organized and licensed in a way that fails to enable the collaboration necessary for success.<br /><br /></div><div><span class="Apple-style-span" style="font-weight: bold;">Glass Cathedrals</span></div><div><br />That's the main point of this blog entry, but I have a little more on my mind related to this and commercial dual licensing and the GPL license, and of course the dozens of variations on the license.<br /><br />In general I agree with the ideals behind the GPL and the philosophy of Richard Stallman and others he works with on this. The idea of making software free and keeping it free is great, especially for certain types of software like infrastructure software that is used en-masse but not frequently customized (ie the improvements are nearly always generic enough to be shared). The reason that I originally pushed for the MIT license, and later the Apache License 2.0, is that these are more commercial and proprietary derivative work friendly licenses. When creating software that is meant to be customized it wouldn't be wise to legally or technically encumber those efforts.<br /><br />What I really wonder is what Richard Stallman thinks about the practice of dual-licensing using the GPL and a commercial license. For these sorts of so called "open source" projects the only reason they want to use the GPL is its terms are the most onerous and are most likely to lead people to pay for a commercial license. In my opinion this is a deceitful corruption of valuable open source principles.<br /><br />The development on these products (not really open source projects...) is generally done by a small group of paid designers and developers and there is little community interaction. Many choose to collaborate only financially instead of bringing their unique requirements and ideas to the table to enhance the core project. There isn't a lot of incentive to try to contribute back or collaborate when it is made clear from the beginning that there is one player who wants to own everything and that you don't really have a chance of being that player or competing with them in their own game.<br /><br />The code becomes more important than the community. It's a bit like putting the cart before the horse in that there is a failure to realize the cause and effect and an attempt to get the affect (the code) with the cause (the collaboration in the community).<br /><br />This is a major case where the use of the GPL breaks down collaboration instead of enabling it. This does happen in purely open source projects as well though, with GPL and other licenses too. Eric Raymond wrote a lot about this with his parables of cathedrals and bazaars. With cathedrals collaboration is limited and that is the biggest problem, whereas the collaboration possible in the bazaar is the real power in the model.<br /><br />I would argue that this happens even with publicly developed open source projects if there is no incentive for or focus on enablement of collaboration, ie no real community that one can become a part of and influence and improve by just being positively involved. Some purely open source projects are this way, but most open/commercial dual-licensed products are this way. Even the "open source" parts are developed in a sort of "glass cathedral". People can see what is going on, and if they really want to they can walk in and participate, but most choose to stand outside and watch and in fact pay to not get too involved.<br /><br />So, in closing, here's to Glass Cathedrals and the opportunity they give to community driven open source projects to differentiate ourselves. It also brings to mind that old saying... something about stones and those who live in Glass Cathedrals....<br /><br /></div>David Joneshttps://plus.google.com/100501146496217472449noreply@blogger.com7