trying stuff with code
There is a small amount of gold inside the Airbag sensor. The sensor contains a small gold-plated marble, which amounts to some tiny amount of gold.
One kilogram of chips contains 1-8 grams of gold (according to Archimedes Channel on YouTube). Judging from the video they look for gold in the processor-type chips (CPU, DSP, etc).
These guys found that road-side dust contains 1000 DKK worth of platinum per 1 ton of dust.
Cody explains that this concentration of platinum equals the amount found in good-quality platinum ore.
In a word, yes. The user Jafromobile commented that a car is likely to eject platinum particulates into the exhaust stream when the catalytic converter is the hottest, when the exhaust system is under pressure, and both of these conditions are basically achieved at the same time. In other words, at full throttle.
Where do cars use full throttle? In an urban area it could be at traffic lights. Other sections of road might in a similar manner reveal consistently-high platinum deposits.
While I intuitively like the concept of Urban Mining - we have all these precious raw materials within easy reach of robots - I know nothing about it. I want to remedy that with the best ignorance cure of all: YouTube! Please, see an exteneded list of videos at the bottom of this post.
The rest of the post will focus on steel recycling to understand the problems and potentials (with regards to AI and robotics) in this industry.
ELB in Duisburg:
Hammel metal separation:
The steel scraps produced in the recycling videos go to steel plants. To see how important this raw material is, check out Thyssen-Krupp's product website.
Here is a list of problems that are specific to steel recycling -- as shown in the videos from ELG.
Perhaps the elephant in the room is that recycling can be a very toxic process. This gives an edge to countries with more relaxed regulation, e.g. certain Asian countries. For example, to extract gold from electronics you may use cyanide. If robotics are more pervasive in robotics, this could eliminate the edge of low-regulation countries as recycling can be done in pure machine environments.
The scrap buyer at the German company (ELG in Duisburg) must buy deliveries of scrap metal on a daily basis and at the right price. For example, nickel can be particularly expensive. She will sometimes go out of the office to buy even small quantities of steel from scrap dealers, and always carry a magnet in her pocket to test the quality.
The dealer has to sell her the steel at a loss because he is running low on space! However, since many competitors also wants to buy the scraps, the buyer has to offer fair prices and have the dealers interest in mind.
The scrap company must examine the quality of a shipment (e.g. steel) before it can be sent to a plant. The process involves cutting out samples, melting the samples and finally determining the alloying elements (e.g. molybdenum) it contains. High-grade steel must have the right mix of non-iron elements. Furthermore, the scrap should not contain toxis elements, such as heavy metals.
A shipment of steel scraps may contain copper or zink parts. These metals would contaminate the steel melt later on, so they have to be picked out of the pile before shipping to the steel plant. Workers at a German plant manually sort through the pile and test the metal with a magnet!
Big pieces of scrap larger than 500 kg have to be cut into smaller pieces, otherwise they won't fit in the furnace of the customer. ELG uses a big crane fitted with a giant claw to cut big pieces of sheet metal (e.g. from dismantled factories) into smaller pieces.
Sometimes ELG uses external companies to reduce the piece size for them.
Another problem that is exposed in the video is that of transportation. A delayed shipment of scrap metal to a steel plant can cause critical downstream delays in the supply chain. In the video, the recycling plant receives too few rail cars into their on-sight train yard (how cool is that!). They have to act fast in order to get the shipment out the door, which means they have to get more train cars fast. Perhaps an alternative would be to ship the metal using self-driving train cars that can be summoned on demand or some other sci-fi solution.
The scrap is melted into new steel in an electric arc furnace at the plant. The furnace uses a lot of electricity! At the plant, the composition of the scraps is analysed again using a randomly selected sample of the shipment. The balance of chromium, nickel and phosphorus to iron has to be within certain bounds.
How could robots solve the problems outlined above?
Some of these videos are about extracting gold from electronics.
I have adapted an example neural net written in Python to illustrate how the back-propagation algorithm works on a small toy example.
My modifications include printing, a learning rate and using the leaky ReLU activation function instead of sigmoid.
import numpy as np # seed random numbers to make calculation # deterministic (just a good practice) np.random.seed(1) # make printed output easier to read # fewer decimals and no scientific notation np.set_printoptions(precision=3, suppress=True) # learning rate lr = 1e-2 # sigmoid function def sigmoid(x,deriv=False): if deriv: result = x*(1-x) else: result = 1/(1+np.exp(-x)) return result # leaky ReLU function def prelu(x, deriv=False): c = np.zeros_like(x) slope = 1e-1 if deriv: c[x<=0] = slope c[x>0] = 1 else: c[x>0] = x[x>0] c[x<=0] = slope*x[x<=0] return c # non-linearity (activation function) nonlin = prelu # instead of sigmoid # initialize weights randomly with mean 0 W = 2*np.random.random((3,1)) - 1 # input dataset X = np.array([ [0,0,1], [0,1,1], [1,0,1], [1,1,1] ]) # output dataset y = np.array([[0,0,1,1]]).T print('X:\n', X) print('Y:\n', y) print() for iter in range(1000): # forward propagation l0 = X l1 = nonlin(np.dot(l0,W)) # how much did we miss? l1_error = y - l1 # compute gradient (slope of activation function at the values in l1) l1_gradient = nonlin(l1, True) # set delta to product of error, gradient and learning rate l1_delta = l1_error * l1_gradient * lr # update weights W += np.dot(l0.T,l1_delta) if iter % 100 == 0: print('pred:', l1.squeeze(), 'mse:', (l1_error**2).mean()) print ("Output After Training:") print ('l1:', np.around(l1)) |
In this post I will give a brief overview of my father's (Georgios Kefaloukos) many activities in the early part of his life. After dinner today I decided to ask him about his life story, and took as many notes as I could. My writing may seem like a glorified enumeration of his many occupations with only sparse attention payed to artistic style.
A young carpenter and waiter on Kalymnos
When my father was 13 years old, he lived alone with my grandmother in Pothia on the Greek Island of Kalymnos. My grandfather had left the family a few years earlier for reasons I don't really know. As a young teenager my father worked as a carpenter in the summer. He also served coffee and cake to customers in a cafe that was located in the hotel in the harbour. This is also where my grandmother worked as a cleaning lady. Needless to say, they did not have a lot of money.
a>
The port of Pothia on the Greek island of Kalymnos.
From petroleum lamps to electric light
As a 14-year-old teenager my father worked as an electrician apprentice in Pothia. The island had just started to transition from petroleum lamps to electrical lighting and many houses needed electrical installations. This meant that it was easy to get work.
Blasting mountain roads with dynamite
As a 16 year old, my father helped build the winding road that runs along the sea from Pothia to Vathi. The work required blasting the mountain away with dynamite. Before the road was built, you had to travel from Pothia to Vathi either by ship or walk over the mountain. In the old days, horse carts would carry passengers from the harbour in Vathi and into the valley where fruits and olives are still grown today.
With the money that my father earned from the roadworks, he purchased a "naftologio" for perhaps 1,000 drachmer (amount from memory), which gave him the right to work as a sailor in the commercial navy of Greece. In Greece, it was, and partially still is, required to pay money to obtain a licence for many types of work, for example to be a cab driver or to operate one of the street kiosks. I have heard that such licences can even be passed down from one generation to the next and are considered valuable assets.
Ahoy, sailor!
During the winter of 1964, my father's 16th year, he worked as a cook for a couple of months on board a ship called Manos. The ship was a small freight carrier that transported goods between ports on the Greek Islands and the Greek mainland. While he sailed on this ship, my grandmother and grandfather moved back together on Rhodes, after many years of separation. They lived in Ippodamou 44 in the old town of Mandraki, which is situated close to the old Turkish bath house of Yeni Hamman. My father left the ship quite soon after and the captain was not very pleased with him.
In the summer before he turned 17, he worked on the cruise ship Ellas that sailed in the Eastern Mediterranean. The ship called port in Beirut (Lebanon), Limasol (Cyprus), Alexandria (Egypt), Rhodos (Greece), Piraeus (Greece), Corfu (Greece) and Venice (Italy), were the wealthy tourists disembarked to explore the historical cities.
Hatha Yoga and spirituality on Rhodos
After his first stint as a sailor, my father became a hippie and settled for a while on Rhodos where he studied yoga while living at home. Perhaps his Yoga practice and his search for an identity prompted his idea to become a monk.
Monk on Athos
At the age of 17, my father met Nektarios, a monk-priest from Kalymnos, in Athens. He told him that he would like to become a monk and they agreed to live together in Agios Minas, which is one of the most remote settlements on Athos - situated at he very tip of the peninsula. My father lied to my grandmother and told her that he would sail with the ship Anna-Maria, named after the Danish princess, but instead he secretly travelled to Athos to meet with Nektarios. Upon arrival he sent my grandmother a letter from Athos and told her the truth. He lived for about 1 year in Agios Minas. Later he would return many times to Athos and learn the art of icon painting.
My Father as a monk (novice) on Mount Athos around 1965.
In his later days, he has travelled frequently to Athos and toured with several Danish celebrities and TV-presenters, including the cook and entrepreneur Klaus Meyer and the journalist Hans Bischoff.
My dad revisiting Mount Athos together with friends in 2011.
Electrician again
After his first stay as a monk (novice) on Athos my father would return to work as an electrician on Rhodos, mostly in hotels. At that time he was 18 and 19 years old. Soon he would join the navy as part of his military duty.
From special forces to ordinary marine
As a 20-year-old my father serving for 28 months in the Hellenic Navy. Initially, he trained as a frogman, but soon had to transfer to other duties for political reasons. The military junta ruled Greece in those years and my grandfather was a communist at the time. The junta didn't allow sons of black-listed people to serve in the special forces. He transfered to the destroyer Aetos (Αντιτορπιλικο Αετός), which is featured in the Greek movie Alice in the Navy. The movie features the actress Aliki Vougiouklaki, with whom the Greek Prince Konstantinos allegedly had a brief love affair. During the final months, my father served on-board a mine sweeping vessel.
Navy photo from the time when my dad served on-board the vessel Aetos.
Archaeologist assistant
???
Sailor again on the seven seas
At the age of 22, my father returned to Rhodes. After working once again as an electrician, he would take work aboard the freight ship Atlantic Star. To rendevouz with the ship he flew from Athens to Tokyo, Japan and waited a couple of weeks for the ship to arrive. To get to the right port, he would travel through Japan on board the high-speed trains from Yokohama, to Osaka and Nagasaki. On-board the Atlantic Star he sailed from Japan to Perth, Australia and from Goa in India to Cape Town in South Africa.
On other ships my father would sail to Sierra Leone, New York, Boston, Murmansk, Northern Norway, Montpellier, Casablanca, and Liverpool. During a trip from Murmansk to Norway, a welder from Rhodos had accidentally opened a big hole in the hull of the ship, which took in a lot of water and listed dangerously in the freezing artic waters. My father told me that he calmly played his guitar while the rest of the crew was in panic.
Drummer in Greek night clubs
At the age of 24, he returned to Rhodes and played drums, guitar and bass at the nightclubs there. It was during this time that he met my mother, who is from Finland.
Black belt in Denmark
As a 28-year-old my father moved to Denmark with my mother. They lived at Saxhøjvej in Valby, Pension Norden in Nørrebro, Rosenørns Allé and Danas Plads on Frederiksberg and finally Lupinvej in Vanløse. In those early years in Denmark he trained Shotokan karate on Amager where he got the black belt.
I work in the ad tech industry, which means that I track people online for a living. Mainly, I do it because the industry has interesting computer science problems and because the job pays well.
I will not defend ad tech. Mainly because ad tech is not important enough to humanity to defend. However, I do believe that ad tech’s algorithms are important to humanity because they can be applied to important areas, such as your health, personal finance and education. However, I have a different point today.
I have a subtle point about privacy. I have noticed that at no point does the ad tech industry need to know who you really are. Ad tech does not need to know what your real name is, what your parents real names are, your actual street address or any other piece of information that identifies you as you to another human being. It is a little bit hard to explain, but I will try. Ad tech is powered by algorithms and these algorithms operate in an abstract space where your true identity is not important. Most ad tech knows you by a random number that was assigned to you. All your interests are also represented by random numbers. The place you live yet another. Ad tech algorithms only care about the relationships between these numbers, not what the numbers actually represent in the real world.
Here is how it works. You get assigned a random number, e.g. 123, to represent you. Then, ad tech will attempt to link your number, 123, with the numbers of boxes that represent products or services that you might be interested in. For example, a box A could be people who need a vacation and box B could be people who could be tempted to buy a new BMW. Ideally, if you really need a vacation and someone really wants to sell you that vacation, then a connection between 123 and A should be made. From ad tech’s perspective, the number 123 is linked to the box A. The algorithm does not need to use labels like “Alice Anderson” or “Bob Biermann”, because the numbers 1 and 2 will get the job done just fine -- from a mathematical point of view.
At some point your true identity becomes interesting, long after ad tech has left the scene. At some point, somebody (e.g. a human being or a robot) might need to print your real name and street address on a card box box, put the product you ordered inside and ship it via DHL. Up until that exact point, your name, street address or any other personally identifiable information is utterly unimportant to anybody. Nobody cares and no advertisement algorithm needs to know. I think this is an important point.
Ad-tech algorithms, if not ad tech itself, can have a massive and positive impact on areas of life that you probably care about. For example, algorithms can help you with your health, personal finance, insurances, education, whether you should buy Bitcoin or Ether today, or whether you should attend job interview A instead of job interview B today, or your kids attend school X or Y. In these areas, relatively un-altered algorithms from ad tech can help. It is important to keep in mind, that again no algorithm needs to know your name in order to work. Not even if that algorithm is looking through your medical record and correlating your stats with the stats of million of other patient records.
Of course it is true that your real identity can be learned from seemingly anonymised data. It might even be fairly trivial to do so, using good old detective skills. Differential privacy has some fairly hard results in that area. However, the main point is that someone has to make a conscious decision to look into the data on a mission to find you and possibly design a new algorithm for that purpose.
Now I get to my main point. Yes, ad tech CAN know who you are with some detective work. However, ad tech does not NEED to know who you are in order to work. This is so important because it means that we can potentially harness the power of algorithms in areas of life that matter — without compromising the privacy of anybody. It is not going to be easy to obtain the granular and self-controlled privacy that is needed, but it is worthwhile. And that is why I joined ad tech in the first place, because the computer science problems are interesting and important — and well, interesting and important things tend to pay well.
I love dal. Here is my simple recipe, which draws it's inspiration from both India and the Mediterranean Region.
Ingredients:
- 200-300 g split red lentils
- 1 onion
- 2-3 cloves of garlic
- 1 can of crushed tomatoes
- coriander seeds
- cumin
- fresh mint
- fresh coriander
- water
- salt and pepper
Fry finely-chopped onion and garlic in olive oil in a pot. Rinse split red lentils in water and add them to pot. It is ok if it fries a bit with the onion and garlic while you get the next ingredient ready. Add a can of chopped tomatoes. Refill the empty can with water and pour into pot. Refill can and pour into pot one more time. Bring it to a boil and then turn down the heat. Add some crushed coriander seeds and cumin. Also add some salt. Boil on low heat until the the water has been absorbed and the lentils have become al dente. Grind some black pepper into the pot and turn off heat. Chop some fresh mint (not too much because the taste can become overpowering) and some fresh coriander and add to pot. Let is settle and cool down a bit. Now you eat it with bread. Enjoy.
I like to dabble with machine learning and specifically neural networks. However, I don't like to wait for exorbitant amounts of time. Since my laptop does not have a graphics card that is supported by the neural network frameworks I use, I have to wait for a long time while my models git fitted. This is a problem.
The solution to the problem is to get access to a computer with a supported Nvidia GPU. Two approaches are to either get my own rig or rent one from Amazon. Which is cheaper?
I will assume that I will train models on my machine (whether local or at Amazon) for two hours every day.
The Amazon p2 range of EC2 machines come with Nvidia K80 cards, which costs about 50.000 DKK. Already this analysis is going to be difficult; I will not buy a computer that costs 50.000 DKK just to train NN models. So, in this analysis I will be comparing apples to oranges, but that is how it is.
The p2.xlarge EC2 instance has a single K80 GPU, which is at least as good as any rig I would consider buying.
The on-demand prie is $0.9/hour; the spot price about five times cheaper. Usage for two hours every day for a whole year costs 4.500 DKK for on-demand and 900 DKK for spot instances. However, the p2 instances is sometimes unavailable in the European spot markets.
What is the best GPU to get for a DIY machine learning rig? In 2016, Quora answers suggested that the Nvidia cards Titan X and GTX980TI would be best. Let's go with that.
This is quite a bit more than 4.500 DKK and that is only for the graphics card. The finished rig would probably cost around 15.000 DKK (Titan) and 10.000 DKK (GTX).
The electricity also has to be factored in, plus that the cards are basically slower than the K80.
With increased usage the DIY approach will become cheaper than Amazon, albeit still a slower option. With usage of 5 or 7 hours/day the DIY approaches break even after a year.
This guide outlines my first (and failed) attempt at mining Ether on AWS. I will first show how I set up a GPU instance to mine Ether. Then I will conclude on the profitability of the whole endeavour in a simple way, i.e. profit = [value of generated Ether] - [AWS cost].
The guide is based on this, this, this and this.
Specs:
| Machine | p2.xlarge (spot instance) |
|---|---|
| AMI | ami-2cbf3e44 |
| Miner | cpp-ethereum |
| Pool | ethermine.org |
| Result | -$0.19/hour |
Log into AWS in us-east-1.
Launch an p2.xlarge EC2 machine. Set the bid price to $0.9. Use the image ami-2cbf3e44, which comes with the NVIDIA drivers etc. The instance should have 100gb disk space to hold the blockchain.
(SSH into the machine)
Install Ethereum:
sudo apt-get install software-properties-common sudo add-apt-repository -y ppa:ethereum/ethereum-qt sudo add-apt-repository -y ppa:ethereum/ethereum sudo add-apt-repository -y ppa:ethereum/ethereum-dev sudo apt-get update sudo apt-get install ethereum |
Start a screen session:
screen |
Run geth in the screen session, which begins to download the blockchain:
geth |
You can now detach from the screen session by hitting ctrl a and d and disconnect from the machine. Periodically check back on the progress of geth downloading the blockchain.
(After a while)
Reattach to the screen session and check the process:
screen -r |
You will see an output that looks like the output below. When geth has caught up with the block chain, you will see messages with 1 block per line.
(When block chain has caught up)
Now we will mine Ether using the default ethminer. This is not recommended according to etherminer.org, but we will see how bad it is.
Install the cpp-ethminer:
sudo apt-get update sudo apt-get install cpp-ethereum |
Benchmark the miner to measure its hash rate. You will need to run it twice. The first time it runs it builds a DAG. The second time it runs it will run the actual benchmark:
# Run twice ethminer -G -M |
I got about 19 MH/s (i.e. 19 million hashes per second) on the p2.xlarge instance.
Create a new screen window by hitting ctrl-a and c.
Now, mine some Ether in the new window. Use the -F option to send it to your account via ethermine.org:
ethminer -F http://eu1.ethermine.org:5555/0x7B194c41B9B5325ae4225Af86Ba4a3a2cdc6Bf4D.rig1 -G |
The long string that begins with '0x7B...' and ends right before '.' is my Ether wallet. You should obviously use your own.
Now, you can detach from the screen session with ctrl-a and d and check back once in a while to see if it is still running. Also check if Ether accumulates in your wallet. Finally, you can check the dash board (mine) to see the progress.
Now, let's tally the Ether that was generated and subtract the cost for the AWS instance.
The AWS spot price for one p2.xlarge is about $0.2/hour. According to ethermine, my rig generates about $0.01 worth of Ether per hour. That gives a negative result of $0.01 - $0.2 = -$0.19/hour.
The AWS p2.xlarge instance would need to be 20 x more effective to break even. Ethermine reports my effective hashrate as 19 MH/s. In conclusion, we need to squeeze at least 380 MH/s out of the p2.xlarge or find something else to do.
Assume that you have a file with some locations as text with one location per line.
For example, here are some school names in Copenhagen, Denmark, stored in schools.csv:
Hyltebjerg Skole Heibergskolen Ellebjerg Skole Katrinedals Skole Peder Lykke Skolen Amager Fælled Skole Tingbjerg Heldagsskole Øster Farimagsgades Skole Sankt Annæ Gymnasiums Grundskole Lykkebo Skole Randersgades Skole Strandvejsskolen Sortedamskolen Grøndalsvængets Skole Sølvgades Skole Skolen ved Sundet Hanssted Skole Holbergskolen Den Classenske Legatskole Tove Ditlevsens Skole Lergravsparkens Skole Vigerslev Allés Skole Bavnehøj Skole Ålholm Skole Langelinieskolen Guldberg Skole Husum Skole Nyboder Skole Vanløse Skole Kirkebjerg Skole Christianshavns Skole Bellahøj Skole Kildevældsskolen Korsager Skole Nørrebro Park Skole Utterslev Skole Skolen på Islands Brygge Brønshøj Skole Kirsebærhavens Skole Rødkilde Skole Vesterbro Ny Skole Blågård Skole Sønderbro Skole Højdevangens Skole Oehlenschlægersgades Skole Vibenshus Skole Valby Skole Rådmandsgades Skole Lundehusskolen Tagensbo Skole
Here is a script, geocode.py, that will attempt to geocode each location in an input stream. It prints CSV output to stdout with the fields input_line, input_line_no, result_no, place, latitude, longitude:
from geopy import geocoders import sys import time import pdb geocoder = geocoders.GoogleV3() SEPARATOR='|' # can also use tab. Comma is bad, since the place will most likely contain a comma. dummy = ['', ['', '']] i = 0 header = ['input_line', 'input_line_no', 'result_no', 'place', 'latitude', 'longitude'] print(SEPARATOR.join(header)) for line in sys.stdin: line = line.strip() results = geocoder.geocode(line, exactly_one=False) or [dummy] for j, res in enumerate(results): place = res[0] lat = str(res[1][0]) lon = str(res[1][1]) out = SEPARATOR.join([line, str(i), str(j), place, lat, lon]) print (out) time.sleep(0.05) i += 1 |
Here is how you might use the script:
cat schools.csv | python geocode.py
Tip: you might want to