Read Latex

Thursday, March 28, 2019

Computing and the Future HW 8 - TRIZ to STRIZ Method

Introduction

Before I answer the homework questions I want to briefly comment on two topics that came up in our previous class:

Boeing 737 Max Catastrophe
The first topic was the Boeing 737 Max catastrophe whose status has largely been addressed by this Seattle Times article. Our in-class analysis using the TRIZ method was very interesting also. 

My comment is this: When I first started thinking about the causes of the two accidents, first of the low-cost Indonesian Lion Air, then the Ethiopian event, I wondered if there was something about the location, the climate, or the ENVIRONMENT that the aircraft were being operated in that could have led to the problem. One example where environment is important is in the case of dissimilar metal corrosion. When two metals of different electrochemical oxidation reduction potential are used in a design, galvanic corrosion can lead to parts failure. A second example for turbomachinery, especially helicopters is sand and grit in the environment. I kept thinking over and over, "What makes the environment of these two crashes different that the typical US and Western European environments?" 

Well it turns out that Boeing CHARGED EXTRA for an additional attitude indicator, a switch and a panel light that enabled pilots to easily turn off the MCAS system when it started ratcheting them into an uncontrollable dive as described in the Seattle Times article.

So the difference was environmental all right, but it was the FINANCIAL ENVIRONMENT in which the aircraft were being operated that was different. I found this realization absolutely stunning - and sickening.

Speaking of Financials
The second comment I wanted to make was on the advent of PredictIt being a useable tool for predicting election outcomes. Prediction Markets are financial instruments that have a 'put your money where your mouth is' validation mechanism. This has gotten me to thinking about finances and cryptocurrencies. This week, Jack Dorsey, CEO of Square took an action that catalysed my thinking on this. The headline is: 'Shorting Banks' and here is my post:


In reality this post may be over-optimistic, but sovereign personal banking may become a long term trend.

Now on with the homework!


1a. TRIZ contains a 39x39 table for identifying things to improve about a particular technology. The 39 rows enumerate conflicts that an improvement tends to create. The 39 columns, which have the same labels as the rows, suggest solution strategies for each improvement/conflict pair.

  • Propose three improvements (table rows) to a technology relevant to your project topic.
  • Identify one or more conflicts (table columns) that each improvement will tend to create.
  • Based on the cells of the table, suggest solution(s) to each improvement and conflict.

The technology I am going to suggest an improvement to is TRIZ itself. I am going to use the 40 TRIZ Principles to overcome limitations in the method. I will make three such improvements:
  • Streamlining the Feature Vector by reducing it from length 39 to 20.
  • Streamlining the Principle Vector, reducing it from length 40 to 23.
  • Streamlining the TRIZ Matrix, reducing it from size 1521 to 400.
Then I will apply the improved technology which I called STRIZ for 'Streamlined TRIZ' to a technology, Machine Learning, which is relevant to my project.
In a way I am using TRIZ to improve TRIZ which is recursive. For the sake of completeness I will cite three worsening features of the current TRIZ, and then with the goal of improving them, find applicable cells in the TRIZ matrix. These three  cases are slightly contrived in that I actually improved the method by using the 40 TRIZ principles rather using than the specific trade-offs enumerated in the TRIZ matrix, but for the sake of using TRIZ to improve TRIZ, that is, for the sake of a recursive view of the problem and also for the sake of fulfilling the assignment requirements I will enumerate the three cases where the matrix could have been used. You can skip these three cases if you want to cut to the case.
Case 1:
Worsening Feature 36:
  • Feature 36 of TRIZ is 'Device Complexity'.
    TRIZ is complex.
Improving Feature 33:
  • Feature 33 of TRIZ is 'Ease of Operation'.
    We wish to ease TRIZ operation.
Solving Principle 17:
  • Principle 17 'Another Dimension'We will reduce the dimensionality of
    TRIZ Features, Principles and Matrix

Case 2:

Worsening Feature 25:
  • Feature 25 of TRIZ is 'Loss of Time'.
    TRIZ complexity makes it time intensive.
Improving Feature 33:
  • Feature 33 of TRIZ is 'Ease of Operation'.
    We wish to speed, and therefore ease TRIZ operation.
Solving Principles 28, 10, 34:
  • Principle 28 'Substitution'
  • Principle 10 'Preliminary Action'
  • Principle 34 'Discarding and Recovering'
    In these cases, logical substitution (as opposed to mechanical)
    is used to simplify TRIZ Features, Principles and Matrix.
    This simplifications constitute a 'Preliminary Action' done
    before the method is employed.
    In the process we discard one version of the method and
    recover another version of the method.
Case 3:

Worsening Feature 35:
  • Feature 36 of TRIZ is 'Versatility/Adaptability'.
    TRIZ complexity reduces its versatility.
Improving Feature 38:
  • Feature 38 of TRIZ is 'Extent of Automation'.
    We wish to automate TRIZ more fully.
Solving Principle 1 and 35:
  • Principle 1 'Segmentation'
  • Principle 35 'Parameter Changes'
  • Segmentation, Merging and Parameter Changes are the Core of
    the TRIZ to STRIZ Transformation

TRIZ to STRIZ 
I want to make a few comments on the TRIZ method and suggest some simplifying transformations that are both semantic and mathematical in nature. I called the simplified TRIZ method STRIZ where the leading 'S' character implies both simplification and streamlining.
The fact that the rows and columns have the same labels indicates that what is an improvement in one context can be a conflict or liability in another. In software engineering, programmers commonly quip that,  "That's not a bug, it's a FEATURE", implying that a bug (worsening feature) in one context can be an improving Feature in another. 

In common engineering/design parlance the Feature space enumerates "trade-offs". Genrich Altshuller called them "technical contradictions". I admire Altshuller for attempting to make the invention process more deterministic. Said again, inspecting this set of phrases reveals design 'virtues' in one context are 'vices' in another. 'Features' is a very significant term in the Machine Learning - the topic I have chosen to apply TRIZ to. More on that after we streamline the method.

TRIZ Feature Vector Transformational Rules --> 39 Labels to 20 Labels

Here are the original 39 Feature Labels that TRIZ uses:


1           Weight of moving object
2              Weight of stationary
3           Length of moving object
4              Length of stationary
5             Area of moving object
6                Area of stationary
7           Volume of moving object
8              Volume of stationary
9                             Speed
10                            Force
11               Stress or pressure
12                            Shape
13          Stability of the object
14                         Strength
15        Durability of moving obj.
16    Durability of non moving obj.
17                      Temperature
18           Illumination intensity
19          Use of energy by moving
20      Use of energy by stationary
21                            Power
22                   Loss of Energy
23                Loss of substance
24              Loss of Information
25                     Loss of Time
26        Quantity of substance/the
27                      Reliability
28             Measurement accuracy
29          Manufacturing precision
30          Object-affected harmful
31         Object-generated harmful
32              Ease of manufacture
33                Ease of operation
34                   Ease of repair
35      Adaptability or versatility
36                Device complexity
37          Difficulty of detecting
38             Extent of automation
39                     Productivity

On disciplined examination the 39 labels can be transformed to 20. This gives us one to bind to each finger and toe! Here are the principles I used for the transformation:
a) Since these features can apply to objects (nouny things) OR processes (verby things) we can use the following substitution chain: Wherever we see the term 'Object' we substitute the term 'Object or Process'. Wherever we see the term 'Object of Process' we can substitute the term 'Entity'. Wherever we see the term 'Entity' we can delete it, with the understanding that it is there. This is similar to the idea used in tensor notation where repeated indices indicate summation, except we are using substitution and deletion. This removes superfluous uses of the word 'object/, which is explicit in 7 of the features and implied in the remaining 32. This transformation is more tersely stated:
Object --> Object or Process --> Entity --> (null)

b) We can eliminate the distinction as to whether an object is moving or stationary for brevity. If it is moving at a velocity of zero it is stationary. This fact when combined with the next transformation cuts the first eight Features to four.
c) We can include changes in the Feature along with the Feature. Mathematically we are lumping the Feature along with its derivatives in the same group.
d) We can denote the Features  of Force as being 'ON or BY' and Harm being 'TO or BY'. This collapses four Features to two.
e) We can merge concepts that have the same dimensions or units. For example stress, pressure and strength all have the same units of force divided by area. Invariably when we Stress an object or process we want to know if it has the Strength to survive it. So this merge makes physical sense via dimensional analysis.
f) Accuracy and Precision can be lumped, provided they are combined with the additional figures of merit, 'Bias' and 'Spread'.
After applying these transformations we obtain the streamlined list of twenty features:

A) 1&2&23&26) Mass and Changes in Mass
B) 3-8,41,42) Length, Area, Volume and Changes in Same
C)  40, 9,40) Position, Velocity, Acceleration
D)        10) Force (on or by)
E)     11&14) Stress or Strength
F)        12) Shape
G)        13) Stability
H)     15&16) Durability
I)        17) Temperature
J)     18-22) Energy Consumption/Production/Changes
K)     24&38) Information Change
L)        25) Time Loss
M)        27) Reliability
N)  28&29,43) Accuracy, Precision, Bias, Spread
O)     30&31) Harm (to or by)
P)        32) Manufacturability
Q)     33-35) Operability, Maintainability, Adaptability
R)        36) Complexity
S)        37) Detection Difficulty
T)        39) Productivity


When we merge table entries for simplification we simply merge their corresponding table entries. The transformation I have proposed reduces 1521 possible entries to 400. We also note that the table is symmetric, it is identical to its transpose. We further note that the principal diagonal is always empty, meaning the original table had a maximum of 1482 cells containing various problem-solving entries and the transformed table had 380 cells. This is a 3.9 fold reduction in complexity of the TRIZ matrix. Completing the transformation according to the rules listed above we have:

Streamlined TRIZ Features - Twenty STRIZ Features


1                    Mass
2              Dimensions
3               Positions
4                  Forces
5       Stress & Strength
6                   Shape
7               Stability
8              Durability
9             Temperature
10                 Energy
11            Information
12                   Time
13            Reliability
14    Accuracy, Precision
15                   Harm
16      Manufacturability
17            Operability
18             Complexity
19          Detectability
20           Productivity



TRIZ Principle Vector Transformation

For brevity I would prefer to stop here, but revision of the TRIZ Feature Vector is only the first part of a three-part streamlining process. For each entry in the Cartesian Product of the TRIZ Feature Vector there are one or more TRIZ Principles that can be invoked to solve the contradiction. There are 40 TRIZ Principles. Let us discover to what degree, they can be transformed and compacted. 

Definitions: A pronym is the original word, an antonym has the opposite meaning of the pronym. This definition is necessary because there is no single-word antonym of the actual word antonym. If this befuddles you as it did me, you can pretend I didn't say it.

Our transformational rules are as follows:
  • When not present add antonym principle to pronym principle.
  • Combine pronym and antonym versions of each principle to form a complementary pair of principles.
  • When antonym and pronyms are already present combine them as complementary pairs.
  • Purge or eliminate redundant Principles.

As before we enumerate the 40 Principles and prepare them.

The 40 Principles of TRIZ Annotated

 
1                                                                Segmentation
2                                           Taking Out / Separation / Removal
3                           Local Quality / Locality / Separation of Concerns
4                                                                   Asymmetry
5                                          Merging / Opposite of Segmentation
6                                      Universality / Combination of Concerns
7                                                   Nested Doll / Telescoping
8                                                      Anti-Weight / Add lift
9                                                     Preliminary Anti-Action
10          Preliminary Action / Like Preprocessing / Suggests Postprocessing
11                                             Beforehand cushioning / Backup
12                                           Equipotentiality / Counterweight
13                                        The Other Way Round / Invert Action
14                             Spheroidality / Curvature Instead of Linearity
15                                 Dynamics / Allow Object or Process to Move
16      Partial or Excessive Actions / Slightly Less or Slightly More of Same
17                                        Another Dimension / Add a Dimension
18                                                       Mechanical Vibration
19                                                            Periodic Action
20                                Continuity of Action / Operate continuously
21                                          Skipping / Operate Intermittently
22           Blessing in Disguise / Eliminate Harm by Amplifying or Combining
23                                      Feedback / Introduce or Modify Amount
24                       Intermediary / Use an intermediary object or process
25        Self-service / Make an object or process serve or perpetuate itself
26                                  Copying / Make lots of copies or simulate
27                 Cheap short-life objects / Disposability / Use Disposables
28      Mechanics substitution / Replace mechanical method with sensor method
29     Pneumatics and Hydraulics / Use gas and liquids instead of solid parts
30               Flexible Shells and Thin Films / Membranes instead of bricks
31                                      Porous materials / Introduce Porosity
32             Color changes / Change the color or transparency of an object 
33                        Homogeneity / Make objects out of the same material
34                                                  Discarding and Recovering
35         Param. Change / State Change, Conc., Temperature, Flexibility, etc
36                                                          Phase Transitions
37                          Thermal Expansion / Suggests Thermal Contraction 
38    Strong Oxidants   / Strong Reactivity / Conc. Variation / Param. Change
39     Inert Atmosphere / Weak   Reactivity / Conc. Variation / Param. Change
40                            Composite materials / Antonym of 33 Homogeneity

We note:
TRIZ 34 is already a complementary pair.
TRIZ 35 subsumes 36 as variants of parameter change/state change idea
TRIZ 35 and 38 are variants of the same idea

TRIZ to STRIZ Transformation


A)    1&2&5) Segmentation & Separation/Merging & Combining
B)      3&6) Separation of Concerns / Combination of Concerns
C)        4) Asymmetry / Symmetry
D)        7) Nested Doll / Telescoping
E)        8) Add or Subtract Weight or Lift
F)     9&10) Preprocess/Postprocess / Preliminary Anti-Action or Action
G)       11) Beforehand cushioning / Backup / Protect
H)       12) Equipotentiality / Gradient / Counterweight / Dead weight
I)       13) The Other Way Round / Invert Action or Order of Process
J)    14&30) Spheroidality / Curvature / Flexible Shells / Thin Films
K)       15) Dynamics / Allow Object or Process to Move or Enforce Stationarity
L)       16) Partial or Excessive Actions / Slightly Less or Slightly More
M)       17) Embed or Project / Add or Subtract a Dimension
N)    18&19) Mechanical Vibration / Periodic Action
O)    20&21) Operate continuously / Operate Intermittently
P)       22) Blessing in Disguise / Eliminate Harm by Amplifying or Combining 
Q)       23) Feedback / Introduce or Modify Amount
R)       24) Intermediary / Use an intermediary object or process
S)    25&26) Self-perpetuating / Copy, serve or perpetuate itself
T)    27&34) Increase/Decrease Lifetime 
                 Make Disposable or Reusable 
                 Discard or Recover
U)       28) Mechanics substitution / Replace mechanical method with sensor method
V) 29&31&32&
   35&36-38&
         39) Parameter Change / State, Color, Transparency, Concentration,
             Temperature, Porosity, Reactivity, Flexibility,
             Thermal Expansion, Contraction, etc,
             of Object, Process, or Environment.
W)    33&40) Homogeneity / Composite / Make out of the same or diff. material

The 23 Streamlined TRIZ Principles
Most of these 23 principles can be suffixed with the phrase,
'object, action, or process' therefore it is eliminated as superfluous. Consolidating these principles for compactness we obtain:


 
1                  Segmentation or Combination
2        Separation or Combination of Concerns
3                        Asymmetry or Symmetry
4               Recursive Packing or Unpacking
5               Add or Subtract Weight or Lift
6     Preprocess or Postprocess Action or Bias
7                   Cushion / Backup / Protect
8                     Add or Subtract Gradient
9                      Invert Order of Process
10      Add or Subtract Curvature or Thickness
11           Constrain or Unconstrain Movement
12                Titrate Actions or Processes
13                 Add or Subtract a Dimension
14             Add or Subtract Periodic Action
15      Operate Continuously or Intermittently
16      Reduce Harm by Amplifying or Combining
17                    Add or Subtract Feedback
18                Add or Subtract Intermediate
19                 Add or Subtract Duplication
20               Increase or Decrease Lifetime
21                         Method Substitution
22                            Parameter Change
23                    Homogeneity or Composite



TRIZ Matrix Transformation into STRIZ Matrix


Again for brevity I would prefer to stop here, but if one more step is taken, the entire TRIZ method is simplified. We are done with the conceptually hard part, all that is left is the horrific bookkeeping transformation necessary to transform the original TRIZ Matrix to the streamlined STRIZ Matrix. To do this we need some mapping vectors that map TRIZ Features into STRIZ Features, TRIZ Principles into STRIZ Principles. Then finally we can construct the STRIZ Matrix.

The original TRIZ Matrix had 1521 cells generated by the Cartesian product of the TRIZ Feature Vector. The TRIZ Matrix contained 4202 Principle references.

Construction of the STRIZ Matrix will be done in three steps:

  • Lumping the TRIZ Principle References into the small STRIZ Matrix
  • Translating the TRIZ Principle References to STRIZ Principle References
  • Eliminating duplicate STRIZ Principle References
  • Sorting the STRIZ Principles and Hyphenating Continuous Ranges
The resulting STRIZ Matrix has 400 cells generated by the Cartesian product of the STRIZ Feature Vector. The STRIZ Matrix contains 2347 Individual Principle References which when hyphenated into contiguous references is transformed to 1559 Principle References. This reduces the size of the TRIX matrix by nearly two thirds. Here is the STRIZ Matrix:


Three Proposed Improvements in Machine Learning
Besides using TRIZ to improve itself, I will discuss briefly a technology relevant to my project in Machine Learning. There are a couple of levels of abstraction that are very important to distinguish. We could improve Machine Learning in General, (MLG) or we could improve Machine Learning in Specific, that is a specific application (MLS).

If we were to improve the MLG case, we might:
  • make development faster
  • make it function with less loss of information,
  • improve its accuracy and precision while reducing its bias
  • make it easier to create, use, repair or adapt 
  • reduce its complexity
  • improve productivity with faster, better and cheaper neurons 
  • reduce the harm of unexplainable algorithms
  • ease development of ML applications

Harm can be done by employing algorithms whose chain of reasoning are not explainable, as in firing someone using a machine-learning program that pares extra employees off a roster.  One area that really needs some attention is the mapping of problems to algorithm in a rapid and efficient way, to wit, ease of development of ML apps.

If we were to improve the MLS case, we might:
  • make execution faster
  • change the topology of the neural network itself, that is, we might change the number of neurons and the way in which they are interconnected. 
  • change the hyperparameters that determine how long the training session will take. These hyperparameters include the learning rate, the neuron type, the basis functions used to model certain features, the number of training sessions or epochs and so forth. 

Since we are asked to pick three proposed improvements, let's choose one from the MLG case and two from the MLS case.

TRIZ for the MLG Case: Faster Development and Harm Reduction


One of the things that makes development faster, as in a Classifier described below, is to train and test, that is train and use, similar neural nets for each output parameter we want to report . This means taking an ensemble of inputs and producing a single value label as output. Said in ML parlance: This means training a set of identical input features to output a single label in a classified learning situation. The only thing that is different is the type of label, everything else is the same. For this we apply the Streamlined TRIZ Principle 1 of segmentation and duplication. We segment by output parameter, and duplicate the networks, specializing each of them to output a specific parameter. This creates the disadvantage that we have all these copies to maintain. We go to the TRIZ table and note that we can consolidate those portions of the network that are the same (Streamlined Principle 1) and segment those portions of the network that are different. (Streamlined Principle 1)

For the MLG Case we choose to reduce the harm of unexplainable algorithms.
Because of the complexity of deep layer neural networks, there rapidly reaches a point where neural nets can effectively classify their inputs, but they cannot tell you how they have done so. This is Streamlined TRIZ Feature 17, COMPLEXITY. This problem is solved using Streamlined TRIZ Principle 17, FEEDBACK in the Tensorflow Playground by showing the input features that each neuron in each layer is processing using visualization of what each neuron in each layer is processing. I will point this out in my class project demonstration.

TRIZ for the MLS Case: Faster Execution

MLS Faster Execution

To speed execution MLS Case we will choose the hyperparameter optimization problem. For this situation we will invoke the Streamlined TRIZ Principle (STP) 19 of copying, self-duplication and self-perpetuation, as well as STP 12 parameter change, and titration. We will make identical versions of the network and run them with different values of learning rate, different numbers of training sessions and different train/test splits. The train/test split is how much of the original data is held separate from the training cases for evaluating or 'testing' how well the neural network functions.

1b. Pick a certain technology (existing or future) relevant to your project topic or any other topic.  Suggest how an implementation of it could "branch out" and do something else. For example, pencils are a technology that "branched out" to also have erasers, storage bins for extra lead, clips for attaching it to a pocket, and so on. []

The branch-out of Machine Learning is exemplified in a strawman proposal I just wrote for an 'Aircraft Classifier'. This tool takes as its input 'features', aircraft geometry, mass properties, power and outputs performance such as top speed, rate of climb and service ceiling. This is a supervised learning problem in that the outputs it gives are labels that correspond to  performance characteristics. Each performance type like range, top speed, rate of climb, service ceiling and so on will require their own neural network. The Classifier can then create interpolated designs that are more optimal than the existing aircraft it is trained on. Combining two such aircraft in a dogfight with a GAN, a Generative Adversarial Network, would allow a more optimal aircraft to be designed. What I'm really interested in is the applied optimal design, the almost real-time design evolution, that AI and ML can facilitate. This constitutes a major BRANCHING OUT that could affect the design of every industrial commodity.

2. For your project, write up 399 words and post to your blog, or do the equivalent in effort on something other than writing and explain briefly.

vTMS™ Progress I have collected most, if not all of the parts for my vTMS™ device and vBrain™ simulator. The vTMS™ work is of three kinds:
  1. Mechanical Engineering
  2. Electrical Engineering
  3. Component Acceptance Testing
Mechanical Engineering Even with this prototype device the tolerances are extremely close - in some cases less than a millimeter makes a difference. For two of the difficult parts, first the rotor, and then the the lamp holder, I utilized 3D printing via ShapeWays. The costs are very reasonable, only eight dollars for the LED holder shown below. The tolerances here are around a hundredth of a millimeter. The lamp holder looks simple enough, but has to allow for cooling of the LED panel, clearance of the wires from the rotor and clamping with adequate structural strength.



I have used Rhino™, a 3D modeler, for  the design work. Here is a typical design image:



Which after some work, leads to a functioning assembly. Using a modeling program saves enormous amounts of time and money. This project would simply not be possible in this time frame without it.


In the process I have found that is is possible to 'supercharge' the permanent magnet (PM) assemblies by stacking them four deep on the rotor. Since PM's are weaker than the magnetic field generated by a traditional inductive TMS pulse I am literally 'stacking the deck' in favor of having an effect. When two rotor faces stick together, it takes many pounds of force to separate them. When the two rotors turn in proximity to each other they induce significant movement in each other. So some technique will be necessary in use to keep them from getting too close to the simulated brain. My rationale for using two rotors is that it is the interaction of the two fields that will allow for precise placement and increase the likelihood of induction of in-brain magnetic fields. Hair, skin, skull, fascia and the three meninges of the brain present a significant magnetic obstacle to a single rotor system. Using two rotors puts a changing field between them.
In addition to the rotor and motor assembly, there is a motor controller, an LED lamp driver, four controls and all the associated wiring. Fitting this unit into its protective housing was the most labor-intensive step so far. Literally millimeters count. You can also notice modeling shortcuts in these pictures. I do not model every thread on the screws or the bevels in the LED plate. This saves time.



Here is a shot of the controls, conveniently on the back. The silver knob powers up the unit which contains a power supply in the base. It turns the LEDs on, which backlight the rotor, and enable power to go to the controller. The red knob controls the speed of rotation with clockwise being faster. The green and red buttons are forward and reverse, respectively. They are momentary switches and must be depressed for the unit to operate. Once the units are operational and stable, the red/green buttons will be replaced with a double pole, double throw switch that leave the unit running without the need to keep holding the switch down. The remaining holes are for cooling the controller, LEDs and heat sink. A gentle breeze wafts through the unit. The motor uses about 3 watts of power. I have measured this using a power supply at 15 V which showed a current consumption of 200 mA. The LED panel is about 4 watts, so the total power consumption is 7 watts. Heating should not be a significant problem. The rotor itself generates a centrifugal flow of air.


Electrical Design
The electrical design is just as important as the mechanical design. It was roughed out on paper like this:


This was then transformed using eDraw™ into a clearer diagram. I find the additional rehearsals that come from codifying the drawing help to reduce the errors I make in assembly. This is especially true on a project that has never been done. Planning needs rehearsal to be successful. Here's the cleanup. The control circuitry is separate, omitted here for brevity.


An unlikely ensemble of tools is necessary for the surgery of assembly:



3. Grad students only: continue with the book you obtained. Read the next 20 pages. State what page numbers you have read. Explain what you agree with, disagree with, and how your views compare with those of other reviewers on Amazon or elsewhere. I have moved this answer to my ongoing review of the book, "The Human Race to the Future" a single curated document that is here. In the Spring Break session for this question I read and reviewed five chapters, 15-19, of the book.

Saturday, March 09, 2019

Computing and the Future HW 7 - Cones of Uncertainty, Project Progress, Book Review


1) Consider cones of uncertainty. Pick a year and apply them to the question, "What will it be like  in that year?" If you prefer, you may choose a different question to apply them to.

At first I had trouble understanding what the 'it' in the question, "What will 'it' be like in that year?" On momentary reflection I am taking 'it' to mean 'life', or 'existence' or 'experience' in a given year. These are such broad strokes that I find them too voluminous in possibility to cope with, my brain melts down at the very thought of it! So, with the reader's indulgence I would like to substitute three concrete concepts for the 'it'. Those concepts include the first of the three basic needs of economics, 'food' from 'food', 'clothing' and 'shelter'. A cone of uncertainty needs a time frame, and a unit of advance, so I will choose a logarithmic progression (base ten) of 1 year, 10 years, 100 years and 1000 years.

Food in 1 year:

  • Foodstuff selection will be the same as now (2019).
  • Bread price will decrease by a few percent.
  • Meat price will increase by a few percent.
  • Going to the grocery store by car will decrease a few percent.
  • Home delivery of both raw ingredients will increase a few percent.
  • Home delivery of prepared meals will increase a few percent.
  • Eating out will decrease a few percent.
  • There will be increasing emphasis on healthful ingredients.
  • There will be reduction in gratuitous carbohydrates by a few percent.

Food in 10 years: 
  • There will be new artificial foods increasing selection by 10's of percent.
  • Bread price will decrease by a 10's of percent.
  • Meat price will increase by a 10's of percent.
  • Going to the grocery store by car will decrease 10's of percent.
  • Home delivery of both raw ingredients will increase 10's of percent.
  • Home delivery of prepared meals will increase 10's of percent.
  • Eating out will decrease 10's of percent.
  • There will be increasing emphasis on healthful ingredients.
  • There will be reduction in gratuitous carbohydrates by a 10's of percent.
Food in 100 years: 
  • There will be new artificial foods increasing selection by 100's of percent.
  • Bread price will approach a constant.
  • Meat price will increase by a 100's of percent.
  • Going to the grocery store by car will not exist.
  • Autonomous home delivery of both raw ingredients will be the sole mode.
  • Autonomous home delivery of prepared meals will be the sole mode.
  • Eating out will approach a constant that is less than 2019.
  • Healthful ingredients will be mandated by law.
  • Reduction in carbohydrates will be mandated by law.
Food in 1000 years: 
  • All foods will be compounded in ready to fix, ready to eat form.
  • Individual ingredients will be less commonplace.
  • Going to the grocery store by car will not exist.
  • Autonomous home delivery of both raw ingredients will be the sole mode.
  • Autonomous home delivery of prepared meals will be the sole mode.
  • Eating out will approach a constant that is less than the 100 year mark.
  • Eating Unhealthful ingredients will be a misdemeanor crime.
  • Carbohydrate quotas will be strictly enforced.

2) For your term project [] do an equivalent amount of work on it. Your work can involve applying cones of uncertainty, theories of innovation, or whatever you like. It's up to you.


For the vTMS / vBrain™ Class Demonstration

This week has been several hours of bench work fabricating and assembling both the vTMS™ and the vBrain™components. I am glad this work started early because there is a lot of detail to cover, even for a simple demonstration.  






Positioning the vTMS rotors in the vicinity of the vBrain™ simulator has been one part of the design process. I originally started with an optical bench, where the rotors could slide closer or nearer to the vBrain™ and the rotors could be raised up and down by the lens rods. This provided y (lower and higher in the head), and z (closer to and further from the head) degrees of freedom, but no ability to move in the x direction (fore and aft in the brain case).  I have abandoned this approach and am now treating the rotors as goose-necked lighting fixtures where the 'light' is the rotor and its swirling magnetic field. Preliminary testing indicates this is a more versatile and robust experimental arrangement than the optical bench and will provide all three degrees of freedom in more of a spherical than Cartesian coordinate system. This is more apropos for navigating the landscape of the head and brain. It is interesting that both the optical bench and the gooseneck lamp mounts exploit a 'lighting' analogy, treating a rotating magnetic field as electromagnetic signals.


3) Grad students: Continue with the book you obtained. Read the next 20 pages. Explain what you agree with, disagree with, learned, and how your views compare with those of other reviewers on Amazon or elsewhere.

I have moved this answer to my ongoing review of the book, "The Human Race to the Future" a single curated document that is here. In the session for this question I reviewed chapters thirteen and fourteen  of the book.

Chapter thirteen, "Space Empire - From Mercury to Neptune", explores notions of interplanetary colonization. The book makes a turn towards the more speculative at this point, diving further into the cone of uncertainty.

Chapter fourteen, "Chasing the Future — Spoilsports of the Prediction Game" changes gears examining the fundamental physical limits on knowing. This chapter is very useful for examining and avoiding specious claims that violate physical law, like the Heisenberg Uncertainty Principle.

Thursday, March 07, 2019

Computing and the Future HW 6 Candidate Projects and Innovation Models

1. Project:
  1. If you have not yet identified a project topic, decide on one.
    Describe your project topic and format.
  2. Write 325 words or more for your project.
  3. Decide what would be a good thing to do next on the project. It does not have to be a big thing, but should be something. Describe it.
Answer 1a. Two Chosen Projects

Of the five candidate projects I examined in five previous homeworks I have narrowed the list to two projects. The first is the tensorflow neural network playground demonstration, which I have extended with custom basis functions. It is done and ready for presentation in front of the class on an internet equipped computer.

The second project deals with real neural networks, that is, the human brain. I have read with great interest the book, "The Human Race to the Future", by Dr. Daniel Berleant, especially the section on magnetic stimulation of the brain. I have found myself inspired to the continue an experiment I began in 2006, to build a permanent magnet repetitive transcranial magnetic stimulator, which I am now calling, vTMS. It will be demonstrated with an instrumented scale model I call the vBrain.

Answer 1b. vTMS

My goal is to answer the question, "Can a person sense a changing magnetic field produced by moving permanent magnets in the vicinity of the head?" If they can sense it, does it benefit them? How does it compare with traditional rTMS?

I want to know whether powerful permanent magnets, mounted in rotors can induce a complementary field in brain tissue, in a manner similar to that of more expensive commercial rTMS units.

The project will consist of three components. The vBrain, a gelatin brain simulator housed in a model skull, and the vTMS, which consists of a left hemisphere stimulator and a right hemisphere stimulator and associated controllers.

To insure safely and reduce administrative overhead in demonstrating the concept I have designed a simulated head called the vBrain that will contain an array of magnetic field sensors in the form of miniature compasses. The vBrain is made out of clear gelatin, glycerine and sodium chloride to simulate the impedance characteristics of the brain. The compasses will be embedded in the gelatin.

This will enable me to measure and compare the magnetic field strength of the old unit with that of the new unit and, as an extra, to estimate and compare the magnetic field strength of these units with that of rTMS machines.

I have redesigned the magnet rotor assembly to be more compact, to use more powerful magnets. There are two identical rotor instances, so that there is one unit on each side of the simulated head. This rotor is has been fabricated using 3D printing and delivered. I have ordered and obtained the magnets also. The N52 grade is the most powerful available and two stacks of them must be handled with some care, as they can create a bit of an arm-wrestling situation otherwise. The material is brittle to impact and I have shattered one by letting it impinge on a spherical magnet learning this lesson.




I have selected Pulse Width Modulated DC motors whose speed can be varied from 0 to 600 rpm and, importantly, can be reversed in direction. This will double the potential complexity and intensity of the interacting fields. These motors have been delivered and tested.







There are 6 magnets in the rotor arranged such that their polarity alternates. This provides for the maximum change in magnetic field per unit of rotation. My main concern was that the rotor, made of plastic and 0.18" thick would not deform excessively in the presence of the alternating magnet poles. The rotor disks have turned out to be very strong. The material is rigid and machinable plastic. I am quite happy with how they turned out. I could have had the screw holes and countersunk regions 3D printed. I did those by hand which took an afternoon of careful measuring, drilling and countersinking. The Shapeways 'versatile plastic' material is robust and tolerant to gradual machining. The magnets fit perfectly in each recessed area with a press fit, secured by a drop of clear glue to support the alternating poles.







One revolution of the rotor per second will produce six magnetic pulses. Since 600 rpm is the same as 10 revolutions per second, the rotor will be able to produce magnetic signals at frequencies from 0 to 60 Hertz. This is about 30 Hertz faster than the fastest brainwaves, to provide some experimental margin.

Brainwave frequencies:

  • DELTA (0.1 to 3.5 Hz) The lowest frequencies are delta. ...
  • THETA (4-8 Hz) The next brainwave is theta. ...
  • ALPHA (8-12 Hz) ...
  • BETA (above 12 Hz) ...
  • GAMMA (above 30 Hz)
The motors will be mounted on L brackets which contain adapters for the rotors. A set of six  4-40 pan head screws will connect the rotors to the adapters for flush mounting, so that there is no protrusion on the gelBrain facing side of the rotor.

These components arrived exactly when promised on March 5, 2019 via Amazon.

For a controller I have selected a reversible unit that can supply the voltage and current sufficient to produce one inch-pound of torque for the rotors. I am hoping this will be sufficient torque for two rotors separated by six inches of distance. I am working with 12 Volt supply and the expected current draw will be less than an one ampere, but there is margin for more current if necessary.

The unit I have selected is the Quimat 7-30V DC 10A 300W PWM Speed Adjustable Reversible Switch DC Motor Driver Reversing Switch. These arrived March 5, 2019.

Operating at 12V instead of line voltage enables a battery to be used instead of a power supply connected to line voltage. This increases safety significantly. I am hoping there is enough resolution on the PWM controller to achieve a nearly continuous range of stimulating frequencies. I will be using Strobe tachometer on the iPhone to measure the frequency of rotation. I just installed it on my iPhone and tested it against the 2006 pmTMS unit and it works quite well, freezing the image of the rotor when the correct frequency is selected using a slider.


Answer 1c.
The next thing I am doing on the project is to assemble the rotor components and wire the controllers and begin working on the vBrain. I have ordered and received a medically accurate skull model, and the gelatin will be cast within its boundaries.




To facilitate the casting, I originally intend to trepan the skull model at the apex of the right parietal plate and plug holes at the bottom of the brain case which includes the occipital, left and right temporal floors. I am now leaning towards inverting the skull and doing two 'pours' of the physiological gelatin material. There is a large hole at the base of the skull that facilitates this process, without having to damage the pristine upper surface. I have obtained the magnetic sensors and am working on an electric field sensor using these calculations. The conductance of fresh neocortex is well known. I used the characteristic dimension of 1 inch, since this is the diameter of the stimulating magnets. The result is that a piece of simulated neocortex this size should have a resistance between 355 and 597 Ohms.




2. Theories of Innovation

Pick an industry. Discuss it concisely from the perspective of each of the following theories/models of innovation:

  • Kline and Rosenberg (KR) Model: Market and Technical Forces
  • Abernathy and Utterback (AU) Model: Gas, Liquid, Solid + Transitions
  • Clark and Henderson (CH) Model: System Components vs. Architecture
  • Teece Model (T): Imitation
  • Christensen I (C1) Model: Resources, Processes and Value
  • Christensen II (C2) Model: Sustaining vs. Disruptive
  • Christensen III (C3) Model: Value Chain Evolution


Machine Learning as a Selected Industry


The industry I am picking is Artificial Intelligence, specifically Machine Learning - the revival of a revolution in progress. This revival is being facilitated by open-source software packages like TensorFlow, Caffe, Pytorch and Keras that are developed and run on personal computers, and then deployed and run in the cloud via containers on Amazon Web Services and Google Cloud services. ML is enabling fantastic advances, in parallel, through Big Data processing and the model of neural networks. This is so prevalent that some are repeating Clive Humby's quote that, 'Data is the new oil'. Andrew Ng is comparing the ubiquity of Machine Learning to electricity. ML is becoming a utility, like the phone pole that stands outside, that we take for granted until it stops working. For the Kline/Rosenberg model I compare and contrast the summary presented in class and a comprehensive outline obtained by a careful reading the paper. For the remainder of the models I will use the abbreviated versions presented in class.

Method

Although the Kline and Rosenberg (KR) article is 34 years old (1986) the issues it raises and examples it uses remain quite timely. This article was both pre-internet and pre-personal computer. The authors used examples from transportation and power generation industries. The web and personal computer have sped up the process of innovation, knowledge sharing, acquisition and increase, but the knowledge and conclusions of the paper remain mostly intact.

Kline and Rosenberg (KR) Class Model:
Market and Technical Forces
  • INNOVATION requires multiple inputs.
  • INNOVATION requires feedback. (compound statement cleaved)
  • INNOVATION creates knowledge.
  • INNOVATION is inseparable from its diffusion.
  • The MARKET improves the PRODUCT.
  • The MARKET improves the COMPANY.
KR Paper Outline
  • INNOVATION is controlled by MARKET FORCES
  • INNOVATION is controlled by TECHNICAL FORCES
  • Successful INNOVATION is:
    • 3/4 MARKET Need
    • 1/4 TECHNICAL Opportunity
  • Successful INNOVATION balances:
    • new PRODUCT REQUIREMENTS
    • MANUFACTURING CONSTRAINTS
    • ORGANIZATION SUSTAINABILITY
  • Successful INNOVATION demands right combination of:
    • affordable COST
    • technical PERFORMANCE
    • TIMING of introduction
    • rapid response to FEEDBACK
  • Canonical Examples:
    • Solar Energy had to wait for costs to drop
    • Concorde cost 15x per passenger mile
  • Models of INNOVATION
    • The Linear Model
      • Research, Development, Production, Marketing
      • Lacks FEEDBACK LOOPS
    • The Chain-Linked Model
      • Expanded Linear + FEEDBACK LOOPS
    • Radical vs Evolutionary INNOVATION and Support Organizations
    • Bicycle Dynamics is an unsolved problem. (random but interesting)
  • Identification and Reduction of UNCERTAINTY
  • Transition from CHAOS to ORDER (Similar to the Freezing Model)
  • Separates SCIENCE from ENGINEERING
  • Orthogonalizes SCIENCE vs INVENTION/DESIGN
  • INNOVATION is
    •  inherently UNCERTAIN
    • DISORDERLY
    • composed of COMPLEX SYSTEMS
    • subject to CHANGE/MODIFICATION
    • initiated by DESIGN instead of SCIENCE
    • enabled by FIVE IMPORTANT PATHWAYS
      • FEEDBACK that links R&D with Production/Marketing
      • PERIPHERAL links that serve the central INNOVATION
      • long-range RESEARCH
      • creation of new DEVICES or PROCESSES
      • SCIENTIFIC SUPPORT TOOLING and DEVICES
    • affects
      • MARKET ENVIRONMENT
      • PRODUCTION FACILITIES
      • PRODUCTION KNOWLEDGE
      • SOCIAL CONTEXTS
  • Two Major Variables:
    • UNCERTAINTY
    • LIFE CYCLE STAGE IDENTIFICATION
So how does the KR Model apply to machine learning?

The KR Model from class is the most useful starting point here:
  • The MARKET improves MACHINE LEARNING (ML).
  • The MARKET improves the COMPANY.
KR Discussion

The MARKET improves MACHINE LEARNING: Our proxy stand-ins for the MARKET will be Google, Anaconda, Guido and StackOverflow. Google open-sourced and released TensorFlow which gave everyone an advanced starting point for this programming change-of-paradigm. Joel Barker has a saying, "When there is a change-of-paradigm, everyone starts at zero". In the case of ML, the entity Google, a complex and many-splendored thing, provided everyone who used it the equivalent of a Formula One race car instead of a tricycle. The entity Anaconda, by providing a curated tool environment, further facilitated the use of TensorFlow. This phenom is epitomized by the phrase: 'conda install numpy', three words which equip the user with a high quality numerical analysis library nearly instantly. Consider: 'conda install how-to-fly-a-helicopter'. Guido van Rossum, the author of Python, provided the language to the world effectively license free. This terse programming language, which makes white-space an operator in the language, has had explosive growth and relevance to ML. The web utility stackoverflow.com, which curates programming Q&A, has become an integral part of ML software development. If one has a programming error, one can copy and paste that error into Google search, which provides a solution with a high rate of positive outcomes.

The MARKET improves the COMPANY: Google is now recognized as one of the de facto leaders in machine learning. The Google Home Assistant, summoned by the phrase, "Hey Google", takes verbal queries, translates them into text and invokes the search engine and other company services to provide a response in real time. This author has used the service multiple times just in the interval of writing this piece. Anaconda, Python and stackoverflow.com enjoy virtual monopolies on the software and services they provide, without exploiting their user community financially or emotionally.


Abernathy and Utterback (AU) Class Model:
Fluid, Transitional, Specific Phases



INNOVATION occurs in three phases:
  • FLUID PHASE
    • CHAOS and EXPERIMENTATION
  • TRANSITIONAL PHASE
    • STANDARDS Set
    • PRODUCTIVITY Increases
  • SPECIFIC PHASE
    • One Technology Dominates

AU Modified Model:
Gas, Liquid, Solid + Transitions 

In this version the AU Model is modified to look more thermodynamically consistent.
  • GAS PHASE
    • CHAOS and EXPERIMENTATION
  • CONDENSATION TRANSITION
  • LIQUID
    • CONSTRAINTS EMERGE
    • PROTOTYPING and PRELIMINARY DESIGN
  • FREEZING TRANSITION
  • SOLID PHASE
    • Design is FROZEN
AU Discussion

Hardware engineers love freezing their designs by writing them in the stone of silicon. Once the designs are tested, the engineers can go home and play.

Software engineers never freeze their designs willingly, and are continuously tempted to go in and improve (monkey with) the code. This introduces bugs that are sometimes never found. Software engineers are never done, and they spend their weekends looking for missing semicolons and rewriting code that already works.

The evolution of machine learning through these GAS, LIQUID and SOLID phases can be seen through the timelines below.


Thermodynamics of Machine Learning

The picture below shows some key innovations that have taken place over the last seven decades of time, including an 'AI winter' that occurred between 1970 and 1995. In 1986, mathematician John Spagnuolo and I implemented neural networks at JPL, but it never went anywhere because we did not include back-propagation. This winter corresponded to a transition between the GAS and LIQUID phases, but preceded the FROZEN period and the Cambrian Explosion of TensorFlow applications.

Source
Missing from this figure are some revolutionary developments identified by Ray Kurzweil (Markov Models), Yann Lecun, and Andrew Ng, especially the abstraction of back-propagation using the chain rule from calculus that has greatly accelerated development. Paradoxically it was published in 1970 by Finnish master student Seppo Linnainmaa, but it has had its greatest effect relatively recently. Here is a curve showing number of ML Patents over time. Note that patents inform the condensation and freezing processes of the AU model I have modified to make it more thermodynamically consistent.

Machine Learning Patents


Here is a curve showing Machine Learning Queries to Google from 2004 to Present. Activity increased around 2014.
Machine Learning Google Queries 2004 to present (3/2/2019)

Here is a  more recent timeline with attribution for recent innovations including Generative Adversarial Networks (GANS) and Long Short-Term Memory Cells (LSTM's). I think of GANS as the "Good Cop/Bad Cop" of ML, since they do there work by pairing an agent who synthesizes results and a critic who criticizes them until some kind of convergence is obtained. LSTMS are used in Recurrent Neural Networks (RNN's) and learn when they should remember and when they should forget! This is reminiscent of 'logarithmic forgetting'. RNN's are most useful for problems that evolve over time, while CNN's (Convolutional Neural Networks) are most useful for problems that evolve over space, as in the pixels of an image.


Different Laws for Different Phases

A second primal principle that emerged in our rumination of these innovation models is that different rules apply at different phases of the innovation (creation) process. A business lesson from the best-practices department might be that it is bad to use the rules that govern one phase to guide the activities in another phase. Said thermodynamically, gases have one set of rules, liquids another, and solids yet another.

In other words there are different laws that apply as we transition from the gas phase of innovation (brainstorming, pulling ideas from the ether), to the liquid phase of innovation (selecting which ideas we are going to run with) then finally to the solid phase of innovation (creating and freezing the design). This suggests that if we wanted to devise a reversible innovation process we could melt the solid and boil the liquid. 

Clark Henderson (CH) Model:System Components vs. Architecture

Two Forms of System INNOVATION
  • Improvements to SYSTEM COMPONENTS
  • Changes to SYSTEM ARCHITECTURE
Applying the CH Model to Machine Learning we can make a few comparisons:

Improvements to Machine Learning Components

ML Libraries are improving and evolving. When the call signature of a library function doesn't change but the internal implementation has improved we say, "The SYSTEM COMPONENTS have improved." However overarching architectural changes are occuring as well. TF2 - Tensorflow 2 - was announced today with fewer API's and a better implementation of eager execution. Eager execution is a new and different architecture for TensorFlow. The changes in TensorFlow are reiterated with links in the AI singularity chapter of the book review, "The Human Race to the Future".

Changes to Machine Learning Architectures

Originally in TF, ML operations were executed in a lazy-evaluation model. In this form the user would specify a graph of the problem and define the tensors that would flow through the network beforehand. Solving an ML problem consists of training the network and then testing how well it works. In a programming construct called a 'Session' this predefined graph was executed. At that time it was determined which resources were required in a 'batch mode' of operation. Eager execution replaced this - define the graph - , then - run the session in a batch -, with a, "just do it right now" approach. This had the advantages of making AI problems more intuitive to code.


Teece (T) Model

Can a PRODUCT or SERVICE be IMITATED?
  • Companies will WIN MARKET if they have SIMILAR ASSETS:
    These Assets Include:
    • DISTRIBUTION CHANNELS
    • SUPPLIER RELATIONSHIPS
    • CUSTOMER RELATIONSHIPS
    • MARKETING CAPABILITIES
    • MANUFACTURING CAPABILITIES
Machine Learning Companies with Similar Assets

At the highest level, Google and Amazon are locked in a duel to control the Machine Learning Cloud. Hot on their heels is a plethora of entrepreneurs looking to develop the next, 'killer app' for ML.

Early in the development of the automobile there were hundreds of companies vying to control the market. We are in a similar mode now where hundreds of companies at all levels in the enterprise and value chain are competing to contribute AI and ML products. Some companies are creating 'code-free' approaches to AI programming to save time and include non-coding users and enterprises. Due to the growing ubiquity of open source, what it means to compete is changing. It is difficult to develop proprietary solutions when some teenager in their bedroom can duplicate the same capability from existing github examples in a weekend. Github accomplishes instant distribution by connecting customer and supplier in a peer relationship. Google provides the marketing by indexing Github so customers and suppliers can be connected through common interest. Manufacturing is done by the programmers. The number of middlemen is dropping to zero. It is worth nothing that Microsoft owns Github, but they don't appear (currently) to be flexing any of their muscle to control access to customers or suppliers. If they did, github would immediately cease exist as programmers would pull their code and migrate it to some other open-source portal such as BitBucket.


Cristenson I (C1) Model

C1 is a RESOURCES, PROCESSES and VALUES Model (RPV). RPV determines what a company can and cannot do. Definitions:
  • RESOURCES:
    • People
    • Money
    • Equipment
  • PROCESSES:
    • PROCEDURES of getting things done
  • VALUES:
    • PRINCIPLES that determine HOW DECISIONS are made.
RPV Vary in the ease with which they may be changed:
  • RESOURCES - Easy to Change
  • PROCESSES - Moderately Difficult to Change
  • VALUES - Extremely Difficult to Change
Machine Learning RESOURCES are abundant on the web due to the open-source innovations mentioned above. The monetary cost (MONEY) of developing ML solutions is quite low and there is excellent training available for free on youtube and the web, and for very low cost through organizations like Udemy, which are giving the University system a serious run for its money. They do so by being first to market. By the time a University course is offered, AI epochs have come and gone, as have the opportunities for invention and innovation. Equipment is interesting. Users can develop on their own machines in a platform independent way. Windows, Linux and MacOS solutions all look and run the same when environments such as Python and Jupyter Notebooks are employed for development. There is one caveat. Machine learning models are extremely compute intensive to train on large datasets. There are now straightforward PROCESSES in taking models that were developed on PC's and pushing them into the cloud.  Training jobs can be outsourced to Google Cloud Console or Amazon Web Services where users can rent custom hardware for a fee, including the latest nVidia Graphical Processing Units (GPU's) are especially useful for training and testing ML codes. This leaves an opportunity for University, Institutional and Enterprise participation, by subsidizing the cost of training large models which individuals may not be able to afford. Thus there is a potential syzygy between the individual virtuoso ML programmer and the larger organization. Organizations like the openAI are attempting to institute a set of values and ethics to keep AI from blowing up with catastrophes. One group recently refused to release a story-synthesizing AI for fear it could be used to create fake news.

Cristenson II (C2) Model

C2 distinguishes between SUSTAINING and DISRUPTIVE INNOVATIONS. Definitions: 
  • SUSTAINING INNOVATION:
    • INCREMENTAL IMPROVEMENTS to PRODUCT or SERVICE
  • DISRUPTIVE INNOVATION:
    • ADDRESS a NEW NEED
C2 INNOVATIONS affect EXISTING and NEW CUSTOMER bases:
  • EXISTING CUSTOMERS:
    • who demand better PERFORMANCE and are willing to PAY for it.
  • NEW CUSTOMERS:
    •  yet to experience PERFORMANCE in new area.
As seen in the developmental timelines above, ML developed rather slowly and non-spectacularly as SUSTAINED INNOVATION until back propagation of neuron weights (along with Generative Adversarial Networks) was automated and distributed in packages like TensorFlow, PyTorch and Caffe 1 and 2. The result has been a DISRUPTIVE spike in software development, that is piggybacking along the more widespread availability of high performance GPU hardware both for rent and for sale. These software and hardware spikes are driving each other, although the collapse of cryptocurrencies like BitCoin in the last year has impacted the GPU industry significantly.

Cristenson III (C3) Model: Value Chain Evolution

C3 distinguishes between FULLY INTEGRATED and SPECIALIZED companies.

Definitions: 
  • FULLY INTEGRATED COMPANY:
    • Performs all COMPONENTS of a SYSTEM production in-house
  • SPECIALIZED COMPANY:
    • Produces one COMPONENT of a SYSTEM

C3 trade-offs are in INTEGRATION vs SPECIALIZATION:
  • FULLY INTEGRATED COMPANY:
    • One stop shop e.g.
      • Boom Box
      • Integrate the best one can do
  • SPECIALIZED COMPANY:
    •  Chooses partners to create a TOTAL SOLUTION e.g.
      • Component Stereo
      • Outsource solutions others do better
C3 further specifies that there is a hierarchy of consumer priorities when it comes to the purchasing or adoption of new products. According to this slide from D. Berleants university course, 'Information, Computing and the Future', those priorities are:


Fully Integrated vs. Specialized

In the case of Machine Learning, if one had to choose a one stop shop it would be Google first, Amazon second, Microsoft third and Apple fourth with Facebook as an also-ran. If one had to choose a specialized company it would be those who curate or contribute specific libraries or hardware, such as Anaconda for Python libraries (thus the name) and nVidia for GPU hardware. The explainability problem in AI has also led to the emergence of small companies, often from University incubators, that just address that single problem - how to get an ML code to explain to a court what it has done, why it made the selection or choice that it made. It all is sounding a bit free-will and non-deterministic isn't it? Since ML is being used to hire (and possibly fire) people, explainability is an important unsolved problem that companies like Kyndi are addressing. Another specialized area is hyperparameter optimization. Parameters such as the learning rate, what memory cell to use, what topology of neurons to use can all be adjusted to produce more optimal performance. Small companies like Sigopt and university research groups are addressing these more specialized concerns, while the big guns are driving overall progress in the field.

Customer Choice Priority

Addressing these priorities in reverse order is easier:
  • Price: All the ML development tools are free,
    while training large ML models in the cloud costs money.
  • Customization: ML Software is extremely malleable,
    liquid and extensively customizable.
  • Usability: The advent of Jupyter Notebooks has made modular chunks of machine learning as easy to trade as baseball cards.
  • Reliability: Machine Learning models provide results that are fuzzy, often varying by several percent on different training runs. This is different than the cold, hard and high precision determinism of traditional numerical analysis.
  • Functionality: With CNN's, RNN's, and Reinforcement Learning, functionality is high.


Footnote to Innovation Models 

Mutation Generates Knowledge via a Random Search

In class, we discovered, through the application of these theories of innovation, that nature uses mutation as a search algorithm, for generating knowledge from a very primitive and first principles level. Successful versions survive and the knowledge of that successful architecture is preserved in its DNA. This is an amazing result with far-reaching ramifications. One important thing to know is whether or not systems are imbued with some base-level architecture, which is then tailored for its environment by mutation, or whether there is no base-level architecture, but rather things that work and therefore exist, and things that don't work and therefore don't exist. This would be the, "there is no spoon" option.  As we wind back the clock, to before life existed, we get to planetary accretion and stellar evolution. The same question applies to that context. As we back up further and further towards the big bang, we have to ask at what point the rules were articulated that determine how things interact. As my son points out in an existential gasp, "McDonald's logos came out of the Big Bang", which is fairly terrifying if you think about it. Orthogonally, we might wonder if we are living in an oscillating universe, which generates a conundrum which appears in my answer to a Facebook question posed by Deepak Gupta:


So in the inset to the figure above I am basically arguing that there is a copy of our universe that is identical to ours where time is running in the opposite direction. Maybe that's where my Nobel prize will come from.

3. Grad students:
Read 20 pages in the book you have obtained. Explain what you agree with, disagree with, learned from it, and how your views agree or disagree with the reviewers of the book that you are analyzing.

I have been informed this week that UALR does not consider me a graduate student, despite having two master's degrees. Nonetheless I soldier in the hope that this outrage will be corrected. Therefore I continue my detailed review of, "The Human Race to the Future" a single curated document that is here. In the session for this homework I review Chapters Eleven on the "The AI Singularity" and Chapter Twelve, "Deconstructing Nuclear Nonproliferation". These topics complement each other as the lessons learned from one can be applied to the other.