Tonight, I went to attend a local meetup event. The talk was given by a faculty from my own department, Jeff Phillips, an assistant professor working in the area of big data analysis. The topic of his talk is about sketch data structure - data structures which uses a much smaller amount of memory to summarize the represented data.This is a cool technique and I am very interested to learn more.
List storage pools
virsh pool-list --details
List all volumes for a pool
vol-list --details POOLNAME
To create a Qemu guest with a image in qcow2 format, we need to first create a storage pool first.
virsh pool-define-as --name qcow2 --type dir --target /mnt/qemu-img
virsh pool-start qcow2
To create a Qemu guest with bridged network
sudo virt-install -n vm1 -r 1024 --disk path=/mnt/qemu-imgs/vm1.qcow2,bus=virtio,size=5,format=qcow2 -c /mnt/sda4/ubuntu-12.04.5-desktop-amd64.iso --network bridge=virbr0 --graphics vnc,listen=0.0.0.0 --noautoconsole -v
I did my PhD proposal denfense yesterday and now I am one more step closer to finish it. It took a week to just prepare the talk. The talk was long and have 45 slides. However, there are only about 8 slides for each project, since there are 4 projects. The talk lasts about 45 minutes. Because it is so long, everytime I practiced it, I almost lost my voice after I finished. We used a new system that can connect skype and regular phone and we can hear everyone clearly. However, background noise came from time to time and this interrupted my talk several times (quite annoying). During the talk, members in my committees asked many questions. This is probably the best opportunity for them to ask whatever questions they have. And I am glad they did that. For some questions, I did not think I answered them well. The good thing is that most of work was done in collaboration with one of the committee members and they joined in, in helping me to answer when my answer was not satisfying or convincing.I should improve in this aspect.
I am working on virtual machine cloning for security testing for the TCloud project. We are looking for an efficient and transparent virtual machine cloning technique that has minimal impact on the virtual machine to be cloned. After some investigation, we found that KVM supports live snapshot for both memory and disk state while Xen4.4 currently does not support this feature yet but could be patched to do live snapshot for memory state easily. btrfs has very impressive performance in taking snapshots: its performance is the best when compared with qemu-img and lvm (finished in about 50 ms for a 4GB disk file). So, we could possibly use both Xen or KVM for our project. The other requirement from the project is we want to provide this security testing as a cloud service. So, we want to use OpenStack as our cloud platform. We had a really hard time in setting up a Xen+libvirt OpenStack cluster and we finally gave up on this approach since it is not supported well yet (in the group C support). As far as I know, there is no online documentation on this topic, until I wrote this one.
During the weekly technical forum within the NetApp ATG group, a team member presented the trip report for OSDI ‘14 this year. In his talk, he mentioned a paper that pushed me to think more in this line. The paper is about how to estimate the working set of a workload, using less memory and runs much faster. The only related work that dealt with the same problem was proposed more than 10 years ago and in that work, the authors tried to get the exact working set size, with a very high demand for memory and runtime. So, in their work, they tried to estimate it, with a goal to reduce memory demand and runtime. They got very impressive results. Together with other papers, I suddenly realized that it is actually quite common in compute science community, to take this approach: when it is too expensive to get the accurate result, it is usually a good time to think about how to approximate it with much lower overhead.