Posted:

(Cross-posted from the Google Developers Blog)

Today, we are open sourcing gRPC, a brand new framework for handling remote procedure calls. It’s BSD licensed, based on the recently finalized HTTP/2 standard, and enables easy creation of highly performant, scalable APIs and microservices in many popular programming languages and platforms. Internally at Google, we are starting to use gRPC to expose most of our public services through gRPC endpoints as part of our long term commitment to HTTP/2.

Over the years, Google has developed underlying systems and technologies to support the largest ecosystem of micro-services in the world; our servers make tens of billions of calls per second within our global datacenters. At this scale, nanoseconds matter. Efficiency, scalability and reliability are at the core of building Google’s APIs.

gRPC is based on many years of experience in building distributed systems. With the new framework, we want to bring to the developer community a modern, bandwidth and CPU efficient, low latency way to create massively distributed systems that span data centers, as well as power mobile apps, real-time communications, IoT devices and APIs.

Building on HTTP/2 standards brings many capabilities such as bidirectional streaming, flow control, header compression, multiplexing requests over a single TCP connection and more. These features save battery life and data usage on mobile while speeding up services and web applications running in the cloud.

Developers can write more responsive real-time applications, which scale more easily and make the web more efficient. Read more about the features and benefits in the FAQ.

Alongside gRPC, we are releasing a new version of Protocol Buffers, a high performance, open source binary serialization protocol that allows easy definition of services and automatic generation of client libraries. Proto 3 adds new features, is easier to use compared to previous versions, adds support for more languages and provides canonical mapping of Proto to JSON.

The project has support for C, C++, Java, Go, Node.js, Python, and Ruby. Libraries for Objective-C, PHP and C# are in development. To start contributing, please fork the Github repositories and start submitting pull requests. Also, be sure to check out the documentation, join us on the mailing list, visit the IRC #grpc channel on Freenode and tag StackOverflow questions with the “grpc” tag.

Google has been working closely with Square and other organizations on the gRPC project. We’re all excited for the potential of this technology to improve the web and look forward to further developing the project in the open with the help, direction and contributions of the community.


by Mugur Marculescu, Product Manager

Posted:
Today we’re announcing the public release of RE2/J: a pure-Java implementation of the popular RE2 regular expression library.

Although RE2/J is not always faster than java.util.regexp, its running time is always linear in the size of the input. Thus when matching larger inputs, especially against patterns containing a high degree of alternation, RE2/J may be dramatically faster. With a backtracking implementation such as java.util.regexp, it is not hard to construct a pathological pattern whose matcher would take years to run on some inputs, so RE2/J's performance guarantee makes it suitable for use in applications where the pattern is supplied by untrusted users, such as the clients of a web server.

If you are looking for a detailed technical discussion of the motivation for RE2 and RE2/J and the tradeoffs involved, please see “Regular Expression Matching Can Be Simple And Fast” and “Regular Expression Matching in the Wild”, both written by Russ Cox.

RE2/J is used widely by Java projects within Google. In many cases, it can be used as a drop-in replacement for java.util.regexp. We are pleased to be able to make this library available for public consumption.

Please head to RE2/J’s GitHub page to find out how to use it!

By James Ring, Google Engineering

Posted:
This week’s Google Summer of Code (GSoC) wrap up is from Matthew Lechleider at Drupal, a widely used content management system used for building websites.
Hello from the Drupal community. We're an open source content management system used to create some of the coolest websites in the world and we're delighted to highlight just a few of our Google Summer of Code (GSoC) 2014 student projects. Not only did 12 students dedicate their summer contributing to Drupal, but most importantly they had fun. Many of the projects below may sound easy, but Drupal 8’s core architecture is a major upgrade from previous versions forcing many students to engineer code in our new systems.

Most students focused on porting frequently used modules to our upcoming version 8. Students such as Lucian Hangea from Romania worked on a vast array of functionality ranging from porting the Diff module with extensible new options to Jayesh Solanki from India who integrated Disqus comments. It is impressive to see contributions from students who are simply happy to be part of our open source community. We can only hope these talented young software geniuses stick around.

Shivanshu Agrawal from India not only created a working version of the Securesite module in Drupal 8 but also finished the Drupal 7 version port during his GSoC application phase. Thousands of users around the world wondered why Drupal 7 sites could not provide simple password authentication until a student made a simple change pushing the module from beta to stable.

Sachini Aparna Herath from Sri Lanka worked on adding support for schema.org mappings to Drupal 8. Helping websites and systems communicate in a standard way, schema.org enriches content allowing it to properly propagate the Internet. Because of Sachini, Drupal 8 site builders can seamlessly point/click schema.org markup to their sites using the RDF UI she built. Similar projects are available in previous versions of Drupal but not Drupal 8, making this project fairly difficult to accomplish in an early beta version.

Andrei-Marius Dincu from Romania ported XMLsitemap to Drupal 8. This module creates a sitemap which conforms to the sitemaps.org specifications and helps search engines more intelligently crawl a website and keep their results up to date. Not only did Andrei port xmlsitemap.module to our newest beta versions of Drupal 8, he continues to provide contributions to previous versions of the module and even mentored students during Google Code-in.

Our community was invited to send two delegates to the GSoC Tenth Year Reunion event held in lieu of the annual mentor summit. We selected Angela Byron from Canada, one of our top GSoC Drupal alumni, and Chandan Singh from India, one of our best students from this summer. Angie's story of beginning with Drupal’s Quiz module as a GSoC student almost ten years ago and becoming one of the most important people in our community is now legendary. We're also excited about Chandan becoming a rockstar developer pushing Drupal to the next level as a promising new contributor. Beginning by leading development of the Entity Embed module as a GSoC 2014 student, Chandan continues to be actively involved in Drupal 8 core development, became one of our essential mentors in GCI 2014, and has returned to GSoC 2015 as a mentor.

A big thanks to all the students and mentors who helped make this summer a success and of course to the entire Drupal community for their amazing support. Last but certainly not least, thanks to Google for making it all possible. The entire open source community is forever in debt to the gift Google provides us with Summer of Code.

It's never too early to start planning for next year's Code-in or Summer of Code. Join our GSoC group on drupal.org, our GCI group on drupal.org, and chat with us via IRC in #drupal-google on Freenode to learn more.


by Matthew Lechleider, Drupal's GSoC Org Admin

Posted:
Google Code-in, our contest introducing 13-17 year olds to open source software development, wrapped up a few weeks ago with our largest contest to date: 658 students from 53 countries completed a record-breaking 3,236 tasks! We almost doubled the number of student participants in 2014 from the previous year’s contest (337). Working with 12 open source organizations, students wrote code, created and edited documentation, designed UI elements and logos, conducted research, developed screencasts and videos teaching others about the software, helped find and fix hundreds of bugs and many other interesting tasks.


General Student Stats

  • 49.1% of students completed 3 or more tasks (earning themselves a cool Google Code-in 2014 t-shirt)
  • 17.93% of students were female, up from 10.08% in 2013
  • This was the first Google Code-in for 552 students (83.9%)


Countries

In the chart below we display the countries with the most student participants.

This year we had nine countries with students participating for the first time: Azerbaidjan, Brunei, Chile, Jamaica, Kazakhstan, Nigeria, Oman, Saudi Arabia, and Taiwan.

Schools

Students from 397 schools competed in this year’s contest.


The five schools with the highest participation are:
  1. Dunman High School in Singapore for the second year in a row had the largest number of students completing tasks in the contest with 58, up from 20 last year!
  2. Govindram Seksaria Science P.U. College, Belgaum (GSS) in India had 49 students.
  3. Technical School Electronic Systems (associated with Technical University- Sofia) in Bulgaria was in the top five for the fifth straight year with 48 students.
  4. Sacred Heart Convent Sr. Secondary School, Jagadhri in India is in the top five again this year with 34 students.
  5. National College "Aurel Vlaicu" Orastie in Romania had 10 students.


Age of Students

The graph below shows the age breakdown of this year’s students.


GCI 2014 Student Age.png


Mentors and Organizations



  • 203 dedicated mentors from 40 countries (including six countries with mentors for the first time: Belize, Burma, Cameroon, Kazakhstan, Singapore and Uganda) helped guide students through the contest.
  • In true “pay it forward” style, 10 of this year’s mentors were former Google Code-in students. Great job!
  • The three organizations with the most tasks completed by students were FOSSASIA (587), Haiku (435), and BRL-CAD (383).


Every part of Google Code-in was bigger this year than ever before, but the most important part of the contest is that more students were introduced to open source software as teenagers. Congratulations students on all of your hard work! Thank you mentors and organization administrators for inspiring these young adults to contribute to open source software and welcoming them into your communities.


By Stephanie Taylor, Google Code-in Program Manager

Posted:
MapReduce for C: Run Native Code in Hadoop


We are pleased to announce the release of MapReduce for C (MR4C), an open source framework that allows you to run native code in Hadoop.


MR4C was originally developed at Skybox Imaging to facilitate large scale satellite image processing and geospatial data science. We found the job tracking and cluster management capabilities of Hadoop well-suited for scalable data handling, but also wanted to leverage the powerful ecosystem of proven image processing libraries developed in C and C++. While many software companies that deal with large datasets have built proprietary systems to execute native code in MapReduce frameworks, MR4C represents a flexible solution in this space for use and development by the open source community.


MR4C.png


MR4C is developed around a few simple concepts that facilitate moving your native code to Hadoop. Algorithms are stored in native shared objects that access data from the local filesystem or any uniform resource identifier (URI), while input/output datasets, runtime parameters, and any external libraries are configured using JavaScript Object Notation (JSON) files. Splitting mappers and allocating resources can be configured with Hadoop YARN based tools or at the cluster level for MRv1.  Workflows of multiple algorithms can be strung together using an automatically generated configuration. There are callbacks in place for logging and progress reporting which you can view using the Hadoop JobTracker interface. Your workflow can be built and tested on a local machine using exactly the same interface employed on the target cluster.


If this sounds interesting to you, get started with our documentation and source code at the MR4C GitHub page.  The goal of this project is to abstract the important details of the MapReduce framework and allow users to focus on developing valuable algorithms. Let us know how we're doing in our Google Group.

By Ty Kennedy-Bowdoin, Skybox team

Posted:
Today’s Google Summer of Code wrap up comes from Stefan Jaindl at the Catrobat Project, creators of the Catrobat visual programming language that makes mobile development accessible to kids.


The Catrobat Project has participated in the last four Google Summer of Code (GSoC) programmes since 2011. During these years, we have built a strong community with local and international students and received valuable contributions to our codebase. This year, we mentored seven students with great projects. We’d like to highlight a few of those projects.

Pocket Code is our Android app for creating and running programs written in Catrobat. Before GSoC 2014, it wasn’t possible to create a standalone APK from a Pocket Code program without some serious hacking. Roland Urbano created Gradle build automation tasks for Pocket Code, extended our Jenkins-CI platform to automatically build standalone APKs, and wrote appropriate tests. Users can now upload a program and request conversion into an APK, and we manually approve these requests and make the automatic conversion. Roland has continued working on the project after the summer and is adding more features.

The ScratchToCatrobat converter is one of our most important undertakings and exists to fill the gap between the well known Scratch system and our Pocket Code project. It empowers kids to run their self-made Scratch projects on their own phone. We had begun development of the converter prior to GSoC 2014, but thanks to Christian Wutte’s GSoC project we’re much closer to a "feature complete" stage. Christian further developed and adapted the converter to support many of our newer Pocket Code bricks (called blocks in Scratch) and formula operators (used in conditional blocks, e.g., if-bricks). The converter is still in alpha and not yet publicly available, but we look forward to sharing it in the future.

One common programming feature which wasn’t supported in the Catrobat language was support for lists. Bernhard Spitzer spent the summer building support into the Pocket Code IDE for creating lists and performing basic operations on them (e.g. add, insert, delete, replace list item). He also added list functions (e.g. number of list-elements, contain certain list-value, returning of a list item) to the Formula Editor. These are now available as bricks which users can add to their Catrobat programs and it brings us closer to feature parity with Scratch.


All seven of our students successfully completed the GSoC 2014 programme and delivered valuable results. We also want to give thanks for the great Google Summer of Code Reunion this year with very interesting unconference sessions, the trip to the Google Campus, the awesome dinner and other happenings such as the evening at the museum. It was definitely a great opportunity to get in touch with other Open Source projects.

By Stefan Jaindl, Catrobat Organization Admin

Posted:


GoogleSummer_2015logo_horizontal.jpg

Do you represent a free or open source software organization looking for new contributors? Do you love the challenge and reward of mentoring new developers in your community? Apply to be a mentoring organization in the Google Summer of Code program! The organization application period is now open.


Now in its 11th year, Google Summer of Code is a program designed to pair university students from around the world with mentors at open source projects in such varied fields as operating systems, language translations, content management systems, games, and scientific software. Since 2005, over 8,500 students from more than 100 countries have completed the Google Summer of Code program with the support of over 480 mentoring organizations. Students gain exposure to real-world software development while earning a stipend for their work and an opportunity to explore areas related to their academic pursuits during their school break. In return, mentoring organizations have the opportunity to identify and attract new developers to their projects as these students often continue their work with the organizations after Google Summer of Code concludes.


The deadline for applying to be a mentoring organization for Google Summer of Code is Friday, February 20 at 19:00 UTC (11am PST). The list of accepted organizations will be posted on the Google Summer of Code site on Monday, March 2nd. Students will then have two weeks to reach out to the accepted organizations to discuss their project ideas before we begin accepting student applications on March 16th.


Please visit our Frequently Asked Questions page for more details on the program. For more information you can check out the Mentor Manual, timeline and join the discussion group. You can also check out the Melange Manual for more information on using the website. Good luck to all of our mentoring organization applicants!


By Carol Smith, Open Source Team

Posted:
Today’s Google Summer of Code (GSoC) wrap up comes from Daniel Shiffman at the Processing Foundation, the organization responsible for the Processing programming language used in many visual arts applications.


Processing is a programming language and development environment specifically tailored to the needs of visual thinkers and artists. Tens of thousands of people use Processing (often called “p5” for short) for design, performance, animation, cinema, and more. It’s also used in several computer science curriculums for its appeal to visually-oriented learners.

We participated in Google Summer of Code (GSoC) for the fourth time in 2014. The students working with us completed eleven projects, many of which are in active use by the Processing community while others will be part of the upcoming Processing 3.0 release. We’ll highlight just a few of these projects below, but we’re grateful to all the students who took part in GSoC with us.

PDE X for Processing 3.0 (Manindra Moharana)

PDE X is a Processing mode that introduces advanced IDE features like code completion, refactoring, live error checking, debugger and more. Manindra helped bring PDE X to a stable state, allowing it to become the default editor in Processing 3.0. Over 30 bugs were fixed as part of this effort. Manindra also added a few new features, including precise error highlighting using the Wagner-Fischer algorithm, manual control over code completion using Ctrl+Space, and a tab outline popup window.


p5.sound addon for p5.js (Jason Sigal)

Jason created the p5.sound addon for the p5.js library to bring the Processing approach to Web Audio. Its functionality includes audio input, playback, manipulation, effects, recording, analysis, and synthesis. Jason also wrote methods for file input / output and ported Processing's Table / TableRow classes to p5.js.


Contributions Manager for the PDE (Joel Moniz)

The Contributions Manager lets users easily install, remove, and update community-developed extensions from within the PDE (Processing Development Environment). Joel’s work this summer introduced new features to the Contributions Manager, such as the addition, removal and update of Tools and Modes without a restart, a new "examples-package"-type contribution, and highlighting contributions.


Sound for Processing 3.0 (Wilm Thoben)

Wilm began work on a lightweight sound library for Processing in late 2013 and improved on it during GSoC 2014 by adding new features, fixing bugs, and introducing cross-platform support. Sound is built on top of methcla, a C++ sound engine with native bindings for low latency support. Sound provides a collection of sound-synthesis objects, analyzers and effects.


by Daniel Shiffman, Processing Foundation

Posted:
We’re getting ready for Google Summer of Code 2015 and the first step is selecting the mentoring organizations that will take part. Organization applications will open next week on Feb 9th, so we encourage you to think about how your open source project could benefit from participation. Today, we welcome GSoC veteran Federico Capoano from Freifunk to share his enthusiasm with other orgs.



It may be the middle of winter in the northern hemisphere, but we at Freifunk are already fired up about the summer. Organization applications for Google Summer of Code (GSoC) 2015 will open soon, and we’ve been urging other open source projects to take part. We’ve participated in GSoC six times, but last year left us with a new level of passion and enthusiasm for the FOSS movement.

Freifunk is a decentralized organization which is building community networks in Germany. Community networks bring a FOSS-inspired approach to the computing networks we use every day: an open network built, owned and managed by private citizens who decide how they use it and which services to run on it. Our community has been participating in GSoC with other communities like Guifi (Catalunya and Spain), Ninux (Italy), and Wlan Slovenia (Slovenia). Together we are making great strides toward the realization of our dream networks.

In GSoC 2014, we had eight successful projects completed by students and mentors from many different parts of the world: Europe, South America and Asia. We don't think about it so often since our contributors are so busy and concentrated on their own projects, but I think it’s amazing that we had volunteers from all over the globe collaborating on open source software applied to open networks!

After the summer, I had the opportunity to attend the GSoC 2014 Tenth Year Reunion which was an incredible experience. I met my Freifunk friends and so many other talented and skilled people working in the FOSS field with many famous organizations from all over the world! Suddenly, I was overwhelmed with a feeling of excitement and enthusiasm!

We had a chance to meet the people of Google Open Source (Chris DiBona, Carol Smith, Cat Allman and Stephanie Taylor) and had the honour of meeting Linus Torvalds, Alfred Spector, Peter Norvig and Dirk Hohndel. Their talks at the event were incredibly inspiring and encouraging. The unconference sessions were very educational and we had an opportunity to discuss improving our GSoC experiences and even the program itself.

It’s difficult for me to describe the whole Reunion experience in words alone, but I can tell you something that might be valuable for you too: this experience ENFLAMED me. If there is something vital to the FOSS movement, it’s enthusiasm. Without enthusiasm and excitement, we wouldn't go anywhere. Coming home to Europe afterward, we brought all that excitement from our community back with us.

Since then, we’ve been conveying our experience to our friends and motivating people to start new exciting projects and prepare for 2015. I personally talked about GSoC and the #reunion14 at:

  • Ninux.org meetups and mailing lists
  • Roma JS (Javascript devs meetup in Rome)
  • PyRoma (Python devs meetup in Rome)
  • Cineca, an IT consortium for universities that also works with FOSS (the place where I work)
  • Fablab Roma Makers

We’ll also be attending several events this year if you’d like to meet up with us: FOSSASIA (Singapore, March 2015), Wireless Community Weekend (Germany, May 2015), OpenTech Summit (Germany, May 2015), Wireless Battle of the Mesh v8 (Slovenia, August 2015), and Chaos Communication Camp (Germany, August 2015).

Similarly, my German friends have talked and written a lot about our last GSoC experience in the different Freifunk meetups, mailing lists, and the main Freifunk blog.

I would like to use this opportunity to encourage everyone in the FOSS community to participate in Google Summer of Code in 2015 because we all need enthusiasm and excitement in our communities. What are you waiting for? Participate and make the best out of it, you won't regret it.

by Federico Capoano, Freifunk mentor