keithyau’s Incredible 2015

In my past 30 years, I don’t mind getting hands dirty (**apply to my keyboard/software only). Not refuse but I will try to avoid touching hardware (Wood/electronics) ~

  1. the reason is i had a super dad do all stuff for me, from homework to toy to experiments to home furniture …
  2. I think 1 person do 1 good thing already too hard, if i’m good at software i should keep focus…

Although my world not small (no comparison here),  looks like i were keeping away from many good things  due to 1) and 2)

At end of Nov 2015, i made my first DIY IEKA standing desk & MacBook Dock. A good start to try getting my hands raw and training execution power, later on, i will develop more DIY tool ~

IMG_4605 IMG_4603

Hey dad, your son grow up ! After making this, I feel really happy. DIY is King !

My crew and i did lots of stuff in 2015 ~ Remarkable as below

  1. Thanks Byron & Spencer’s trust (if they read the post), we hosted up Coconuts media with 10M+ PV traffic, peak 20k users, SLA around 99.99% up time. Before 2015, it were about 1M-2M PV. Also, thanks editors team bring in traffic so that i can have the challenges, not everyone can meet real traffic.
  2. Kevin Tsang Jacky Chan and I Keith Yau found a team — BootDev, originally from Mobingi, Now focus in AWS/Drupal related community business, pushing Drupal and AWS user group forward in China. This year, we did lots of community works in China & APAC. Thanks AWS China team, prompt me as AWS Community hero ! It is my honor.Screen Shot 2015-12-25 at 2.17.28 PM.png
  3. Congrads Former Mobing get into 500Startups and thanks Wayland Zhang proofed the concept works. I will super charge and make another better thing.
  4. May2015, thanks Drupal Assoc buy me free tickets to LA and join DrupalCon, As a Drupal scholarship winner, i see Drupal and the community in a total different angle.
    IMG_7993
  5. Thanks my crew, Jacky & Kevin~ Travel with me and enjoy this crazy year ! We been to Bangkok, Singapore, Taipei, Beijing, Shanghai, San Fransisco, LA and …. Did lots of presentation and Pushing our community business forward !
    IMG_4046 IMG_4202
  6. May2015 also, First time been to Canada, first time see wild animals in national park and thanks mum & uncle & sisters for everything
    DSC_0134
  7. May2015 also, thanks spencer & his gradma, first time been to San Fransisco, the city of tech, everyone (include me)’s dream ! We visit some companies, getting into a SF local home and supermarket and learn alot ! I will be there ! Officially !
    IMG_7333IMG_7905
  8. We had a Chinese website launched about how we do Scaling as a service. Thanks strikingly for their super tool to let us build a website in a few hours !
  9. Jun 2015, Becky and me’s first relocation to Thailand, we visited lots of apartments and try to rent one ~ Anyway, failed but really nice experience that planned to live outside our country. P.S. I WILL RETRY !
    IMG_3806 IMG_3816 IMG_3823
  10. We organize a drupal tech meetup late Nov and supported by China Accelerator, which were really successful.
    IMG_4554
  11. Drupal 8 released this Nov2015, finally
    IMG_4555
  12. Oct2015, thanks Becky zhang ! Brave girl ! Present us on stage with hundreds of people And get a prize for BootDev !
    IMG_4455 IMG_4450
  13. Nov2015, Got my first TV + PS4 at my own place 48 inch 4K ! Feel sweet and home
    IMG_4526
  14. Dec2015, AWS Shanghai User group launched ! The first China User group ! 851333089_4577116538317424546.jpg
  15. Dec 2015, The best workspace ever in my life !

Hopefully 2016, Coconuts can growth 100M+ traffic and Bootdev can get 500 developers join our both Drupal and AWS user group !

See how far we can go at 2016 ! and remarkable 2015 !

 

 

Advertisements

why BootDev — AWS management (EC2 decrease performance by time)

In our experience, if your EC2 got resource left for a period of time, lets say 1-2 months, even AWS say it is not, but we always experience performance drop. For example: server response time drop from 400ms to 600ms . Or CPU raised from 40% to 70% in average.

In many case, many users will not 100% * 24 * 7  consume their server resources. So, in my experience, AWS want to reduce cost and reallocate some resource privately to other clients. If your server is actually low utilization, it is OK. But, in my situation, we need high response time and we reserve the CPU for that. We will feel performance drop.

How we deal with that ? That is what auto-scaling group should do. If in your auto-scaling group, your server will keep renew itself. Like turning off old servers and start new servers, the EC2 resource allocation will maintain at best performance.

Such issue especially obvious in Drupal. Drupal PHP request involve in many hooks / functions / third party calls etc. Server CPU requirement is higher than other web PHP applications. So, renewing server can help to keep better performance.

How to deploy auto-scaling group ? Call BootDev 🙂

Hope above information can help anyone that feel AWS sometimes drop performance by time 🙂

 

 

 

 

Complete solution to bypass GFW (SQUID + SSH Tunnel + AWS)

I used single solution to bypass GFW for a long time. Running VPN, SSH Tunnel, SQUID … Each of them got pros and cons. And sometimes not smooth. Especially I want to watch Videos that only free licensing in China, but also facebook(ing). VPN cant help.

While the power of GFW increasing, SSH Tunnel & SQUID sometimes will be de-functionated. Recently, I use the combination of SSH Tunnel + SQUID as my solution and works well.

For junior technical people, the idea is a bit complex. What I am doing is SSH encrypt SQUID requests. (Mapping the remote SQUID port to a new created local port underlining SSH)

There is free one AWS EC2 available for 2012 subscription, so I use AWS for my remote server.

SSH Tunnel
localhost:22+————————————-+ 22:remote
  +  +
  + +
   8080+  + 3128 (Remote SQUID on AWS)

 

With this setup, I can use FoxyProxy (Google Chrome) or AutoProxy (Firefox) help to forward all request from browser to local binded SQUID port (8080). So, I can enjoy http/https requests but not SOCK5 and those requests are encrypted, GFW wont drop the request by filtering keywords.

To establish the tunnel above, the easiest way in Windows system is using bitvise. With the attached file setting. (Of coz, with your AWS key and able to login)

Capture

Enjoy

Keith

Email pipe

Hi all,

If you have a email server on Linux server, you can pipe email to a backend script easily.

This allows you to parse email or control server in email.

Take an example for PHP

echo 'test: "php -q |/your/script.php"' >> /etc/aliases

This command will need php cli support, in Ubuntu, enter

sudo apt-get install php5-cli

And then create a file called script.php start with #!

  1. #!/usr/bin/php
  2. <?php
  3. // read in email from stdin
  4. $fd = fopen("php://stdin", "r");
  5. $email = "";
  6. while (!feof($fd)) {
  7. $email .= fread($fd, 1024);
  8. }
  9. fclose($fd);

The email content will be in $email, you can do whatever you like

If you are thinking of email command, add a line like

  1. exec ($email);

If you are thinking of calling Drush without managing drupal init externally, you can 

  1. exec ("drush your-custom-command " .$email);


Of coz, it needs certain privilege.

Enjoy ~

Keith

Drupal Adding apache solr index for field-collection

If images / content are bounded by field collection module, you can try approach below to index to solr

enjoy

 

//Index product images

$field_collection_items = field_get_items(‘node’, $node, ‘field_image_set’);

$field_collection_item_ids = array();

$i = 0;

foreach ($field_collection_items as $field_collection_item){

$field_collection_item_ids[$i] = $field_collection_item[‘value’];

$i++;

}

$field_collection_item_fields = entity_load(‘field_collection_item’, $field_collection_item_ids);

$image_fields = array();

$i = 0;

foreach ($field_collection_item_fields as $field_collection_item_field){

$image_fields[$i] = field_get_items(‘field_collection_item’, $field_collection_item_field, ‘field_image’);

$i++;

}

 

$i = 0;

foreach ($image_fields as $image_field){

foreach ($image_field as $image)

$path = file_create_url($image[‘uri’]);

$document->setMultiValue(‘sm_field_image_’.$i++, $path); //Set multiple field_image values

}

 

Adding index to apache solr by Drupal hook

1. Add index by setMultiValue

function scp_solr_search_apachesolr_update_index(&$document, $node){
$document->setMultiValue(‘sm_field_xxxx’,
$node->field_xxxx[‘und’][0][‘value’]);

2. Add modify query

function scp_solr_search_apachesolr_modify_query(&$query, $caller){
$query->params[‘fl’] .= ‘,sm_field_xxxx’;

3. update solr and done ~!

* scp_solr_search is my module name Enjoy