Pumpkin Pancakes!!!

Source: Pumpkin Pancakes!!!

Pumpkin Pancakes!!!

Made these this morning, came out fairly well 🙂

First I mixed the following in a mixing bowl:

• 1 1/2 x cups of Aunt Jemima (Whole Wheat Blend)
• 1 x large egg
• 1 x cup milk
• 1 1/2 x Tbsp Oil

Next, in a smaller bowl I mixed the following:

• 1/3 x cup of Libby’s Pumpkin Pie Purée
• 1/2 x cup of granulated sugar
• 1/4 x tsp table salt
• 1 1/2 x tsp ground cinnamon
• 1/2 x tsp ground ginger
• 1/2 x tsp ground cloves

Next, blend the pumpkin mix into the bigger bowl.

Finally get a skillet and makes some pancakes!!!

FYI: The mixture will be a little thick, so make a few small test cases first… They will be fluffy if when you flip them over you do not smash them down 🙂

Enjoy!

Theory / Rant: Multi-multi-dimensional Quantum access DNA storage

This is my first late night rant, please bare with me.

So, this topic has been festering in my mind progressively for years now. And until recently I did not really have the time to really sit down and ramble on about it. This theory is not complete on implementation and has many issues, which need to be resolved.

Note: The main technologies that I mention below are real, however the way in which I describe them interacting does not yet exist (that I am aware of).

When I studied Data Structures and Algorithms, I had pasted the definitions, algorithms and proofs of some of the popular and more efficient solutions up on my wall. This was around the year 2000 and I was living in a basement apartment. My thoughts often wandered while I looked at the proofs, thinking that there had to be a better way. My thoughts would take me down a path of deeper abstract and dimensions. So, finally I had thought of a sphere as an abstract model with n degrees (or layers/smaller spheres within. Similar to the Russian toy doll) of complexity of what we now call sharding or scaling; and additionally multiple entry points and paths to the data (this portion of it always sounded great, but I never had worked out the math for such a problem. (quick note: I did wish to use the “shortest path algorithm” for it, but thought it may be a bit premature to introduce it. Routing back then always bothered me as being a real big weak spot for latency. Recently though with cloud computing, you can dynamically call regions, data centers, shards, LBs, servers, databases, , and finally your hash key in order to get a value with a couple lines of code and it would be a fairly straight forward route )

So over the years there has been talk of storage devices being created from nano tech, coffee, Holographic, crystals, and now DNA. I wont get into too many details, but the latest news from DNA research sounds fairly promising. I will say that the last thing I heard was that as of recent 7 grams of DNA would be enough to hold all the world’s data. Here are some of the possible pluses to having this work:

• 1. Electromagnetic forces do not play havoc with the data.
• 2. with respect to our gauge of time, there is no degradation of the information held in this form over time.
• 3. We could do incremental backups of the worlds data daily without worrying about space.
• 4. Electricity used for access would be reduced to something so small we could probably not gauge it accurately every time, which would lead into…
• 5. There would no longer be a heat signature for us to worry about!
• 6. Mirror images (backups) could be created and sent to safer places or two aliens even!
• 7. Last but not least… my favourite part, which allows for the “multi-dimensional” portion of the title, is that DNA can be broken down into smaller parts.

I should probably start explaining what I am carrying on about. Lets briefly define each part to this.
– DNA digital data storage (wiki):

DNA digital data storage refers to any scheme to store digital data in the base sequence of DNA.

– Multi-dimensional access: In terms of data, this means that you need all “n” points of dimensional reference point/s to retrieve the intended-targeted data. So, you plug in 3 points of data and get back one value. This leads into Holographic data storage.

– Holographic data storage (wiki):

Holographic data storage records information throughout the volume of the medium and is capable of recording multiple images in the same area utilizing light at different angles.

– Quantum access: with respect to this theory could range from routes of access, encoding/decoding hashes for data keys as well as data values.

My thoughts: This idea first hit my mind as I had thought about how holographic storage might of worked; that coupled with my original abstract multi-layered sphere concept allowed me to think of this new access arrangement. Also, I would of used multi-holographic in the title, however I am not sure if that tech would be applicable to DNA.

Enough rambling… Here is what I might expect out of this process for this theory:

Creating a Record:

• 1. Some value is passed to a quantum computer (QuC)
• 2. The QuC would, break the value into memory allocation chunk sizes (MACV)
• 3. If MACV in Cache, then grab the corresponding key (MACK) and store in a list (MACLST)
• 4. The QuC would send all new values/data to the DNA digital data storage solution where the data would be written to a new placement within something similar to the DNA cells and/or atomic structures, which would compare to the holographic tech which could store multiple data at different angle at the same placement within a storage container. The circumstances for storing the data would then be sent back as some DNA key mapping (DKM), this DKM would be returned as confirmation of the data being stored.
• 5. The returned DKM along with then MACV pair would then be saved in cache. The DKM would also be added to the MACLST in order.
• 6. When the value has been completely stored a hash is created from the MACLST plus the datetimestamp and would then use something similar to the tombstone process with which Apache Cassandra uses to keep order.

Hope you enjoyed this so far. Please comment and add to this crazyness, more perspectives would probably yield better results 🙂

More to come soon!

All Things Python

This will be a living post regarding all things Python. I will hopefully be covering all of the basics, every engineer should know topics, advanced topics, community topics and standards, 3rd party interaction, and probably a slew of extra goodness :)… and as I write more, then will be added to the mix. Enjoy!

---

Getting Python up and running on your rig:

This is probably not a huge hurdle for most people, but just trying to cover all steps.

• python-2-x-3-x-installation-and-setup

---

Basics:

Coming in the near future…

---

Every engineer should know topics:

In this section I will be describing industry standard topics that are asked in interviews for a junior to mid-level position.

• python-iterators

More coming soon…

---

Linux – Definitions and Commands

Terms
:term: FHS – File system layout (default folder layout)
:term: mounts – locations where storage devices are allowed to reside (:locations: [‘/boot’, ‘/usr’, ‘/var’, ‘/home’])

Commands

still to be listed:
history, time, type, stream, pipe, redirect, STDOUT, STDERR, STDINPUT, mail, tee, xargs, head, tail, cat, fmt, pr, expand, convert, join, paste, nl, sed, sort, split, uniq, wc, …

:cmd: ls
:desc: list directory contents
:options: [‘-a’, ‘-l’, ‘-h’, ‘-t’]
:example: ls [OPTION]… [FILE]…
:usage: $ ls -alht
:usage-desc: list all + long format + human + sortby newest first

:cmd: hier
:desc: shows FHS folder hierarchy
:usage: $ man hier

:cmd: dpkg
:desc: -L shows where the files are installed in your system
:example: dpkg -L <package name>
:usage: $ dpkg -L python3.4

Finding Stuff
:cmd: find
:desc: Searches the specified folder structure, which means it is accurate, but can be very slow.
:options: [‘-name’, ‘-user’, ‘-xtype’, ‘-amin’, ‘perm’]
:usage1: $ find / -name “passwd”
:usage1-desc: searches for a file named “passwd”
:usage2: $ find / -perm /4000
:usage2-desc: searches with permission 4000

---

:cmd: locate
:desc: is a local operating system database lookup, which means its faster than “find”, however like all single instance databases can be out of date. A manual update can be forced by using “updatedb”.
:usage: $ locate <filename>

---

:cmd: whereis
:desc: Search based on binary, source, and man pages
:usage: $ whereis ‘passwd’

---

:cmd: which
:desc: Search based on binaries only
:usage: $ which <package>

:cmd: ps
:desc: report a snapshot of the current processes
:usage1: ps -aug | grep http
:usage1-desc: display the current “http” processes
:usage2: ps -augfx
:usage2-desc: display all running processes in detail

:cmd: ln

:cmd: cal

:cmd: wall

:cmd: du
:desc: displays size of current folder
:options: (of interest) [‘-L’, ‘-l’, ‘–inodes’, ‘-h’, ‘-s’, ‘–total’, ‘–bytes’,’–all’]
:usage: du -hs
:usage-desc: human readable + summarize

Shell Environment

:cmd: env
:desc: displays the current environment variable available in memory
:usage: $ env

:cmd: uname
:desc: print system information
:options: [‘-a’, ‘-s’, ‘-n’, ‘-r’, ‘-v’, ‘-m’, ‘-p’, ‘-i’, ‘-o’]
:usage: $ uname -a

:cmd: echo
:desc: display a line of text
:options: [‘-n’, ‘-e’, ‘-E’]
:usage: $ MYDIR = /usr ; echo $MYDIR ;
:usage-desc: sets the folder “/usr” to the variable MYDIR, then echo calls that variable with a “$” in front of it.

:cmd: bash

:cmd: help

:cmd: set
:desc: Set or unset values of shell options and positional parameters.
:options: set [-abefhkmnptuvxBCHP] [-o option-name] [–] [arg …]
:usage: $ set ; help set | less ; unset ;

Folders

• /boot – all for boot
• /dev – interfaces to devices
• /etc – config files (ascii file type)
• /usr – profile files
• /sbin – binaries for super users
• /bin – binaries for all users
• /lib – 32bit library files for programs
• /lib64 – 64bit library files for programs
• /home – where user profiles and some mounts might live
• /mnt – manual mounts
• /media – auto mounts
• /opt – optional
• /proc – interface to the kernal!
• /run – info about running processes
• /srv – services doc locations
• /sys – interface to hardware
• /tmp – auto trash
• /usr/local – my toolset would go here
• /usr/share – everyone
• /usr/src – source code
• /var – filled with processes that are running, logs, databases, doc root…

Cassandra core concepts broken down

1 of many Cassandra (educational) posts to come.

I am new to Cassandra, but definitely not new to databases and Cassandra’s services that make it so interesting. As I learn more, I will create more posts to expand upon everything Cassandra. I have learned way more than what is in this post already and very excited with the idea of sharing this knowledge… so look out for future posts!

There is a great deal to look at under the hood with Cassandra, but lets start with a good foundation.  In this post I will be focusing on a break down of Cassandra’s core concepts.

So what is Cassandra? The wiki page states that:

Apache Cassandra is an open source distributed database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure. Cassandra offers robust support for clusters spanning multiple data centers, with asynchronous masterless replication allowing low latency operations for all clients

This description sounds fantastical, but what does it actually mean?  Lets try to break it down into easily digestible chunks.  Lets first address all of these industry phrases in bold with respect to Cassandra.  I will also touch upon the “Data model” concept at the end.

Distributed Database:

– This means that data and processes are spread across more than one machine

Great! so the data and processes are spread across many machines, but why do we care?  We care because this means that no single machine (or a “node” as it is referred to in Cassandra lingo) holds all the data or handles all the requests.  Technically speaking its similar to a load balancing mechanism.  Each node is typically configured to house a portion of data (the distributed pieces of this data are known as “chunks”).  Additionally, all requests by design are broken up with respect to how the data was distributed.

Now we are getting somewhere.  So now if your data or processing gets to large within your environment all you have to do is add more nodes.  Additionally, if you need more parallelism, just add more nodes.

Distributed database architectures, built and configured correctly, with respect to Cassandra also means that if a node becomes unreachable that the service itself is still intact and usable.  This type of structure is also known to not have any single points of failure.

Finally, if a distributed Cassandra mechanism is well designed, it will scale well with n number of nodes. Cassandra is one of the best examples of such a system. It scales almost linearly with regard to performance as data is added and when we add new nodes. This means Cassandra can handle a ridiculous amount of data without wincing or exponential degradation of performance like most data storage solutions.

High Availability:

A high availability system means that if there is a failure the client will not notice any disruption.  This is usually achievable by having some sort of redundancy in place, like additional servers, clusters, or data centers.

Multiple data centers:

First, the term “data center” typically refers to a physical location where servers live, however with Cassandra the term is used a bit differently.  Data centers or DCs are more of a designation for a grouping of computers with a directive.  So, you could actually have multiple DCs in one physical data center.

Moving forward, multiple DCs indicates, more than not, that syncing or replication data between the different DCs is occurring.  Reasoning for having multiple DCs could be, but is not limited to replication, quicker regional access, and separation of data processing. With older data storage solutions replication is typically difficult on many levels, however this is a fairly trivial operation with Cassandra.

Replication:

Cassandra’s replication service is extremely powerful.  The replication architecture is referred to as masterless, which means, yep you guessed it, it has no master.  There is also no slave concept either.

Replication in Cassandra is also configurable so that n + m nodes will replicate data, however only m need to be verified first;  This configuration is extremely allows for crazy fast responses especially when replication is global.  Another helpful feature is that the replication is done asynchronously further decreasing latency when verifying that data has been written.

Data Model Introduction:

Cassandra has a three container data model hierarchy, one within another.  Here they are, with their RDBMS counter part terms, starting with the outermost and working our way in:

• A keyspace is equivalent to a database
• A column family is equivalent to a table
• Finally columns which are still reminiscent of columns names, but created and accessed a bit differently. Columns appear to be, visually, contained within a fourth container called rows; and these rows are still identified by a unique key similar to the typical RDBMS primary keys.

So, this raps up the core concepts of Cassandra at a very high level. I am hoping to turn this into a full set of posts that cover Cassandra at all depths.

Hope you enjoyed this post and perhaps learned something.  If you find any of the information incorrect and/or out of date, then please comment.

Thank you