During the semester, I did work on three projects: Butter, Popcorn Maker FCP, and TestSwarm for iOS. The following links lead to each blog post I wrote this semester.

January

• Popcorn Maker – Bug 157
• Popcorn Maker – Bug 67
• DPS911 Release 1

February

• Getting ChocTop to Work
• Popcorn Maker Bugs 256, 257, and 281
• DPS911 Release 2
• DPS911 Release 3

March

• Butter: Bugs 256 and 390
• Butter: Bug 434 Status
• Butter: Bug 459
• TestSwarm for Popcorn on iOS
• DPS911 Release 4
• Butter: Bugs 434, 458, and 493
• Butter: Bug 463, the Undo Manager
• DPS911 Release 5

April

• DPS911 Release 6
• DPS911 Release 7

Lessons Learned

Lesson 1: It’s Not Just About Code

The first lesson I learned was that although code is important, it’s also equally important to be online. Using social media to talk about the work being done and increasing awareness in the community is key to the growth of the project.

It’s also important for the project to have a good structure that is easy to follow. People will be more likely to contribute to code that they can understand.

Second Lesson: Consistency

Working on an open source project requires consistency. Code often changes so quickly that what was working today will likely be broken tomorrow. It’s also important to always blog and use Twitter, IRC, and the other forms of communication used in different projects.

Third Lesson: Plan and Execute

Ideas are good, but planning and execution is more important. When there is an idea that people are unsure of, or one that may cause many changes, breaking it down into smaller chunks and iterating on it can quickly reveal how viable it is.

Fourth Lesson: Organization

Being able to track what needs to be done, the state of each task, and who is doing what is extremely important. This is where Lighthouse and GitHub come in. Lighthouse not only keeps everything organized, but we can look at how long, on average, tickets take to complete, how long reviews take, and how often bugs delay estimated completion times. All of these details can help the planning of the project in the future: better estimates, and a better a idea of what has worked and what hasn’t.

Reflections

I am very glad that I was able to take this course and work on Butter, Popcorn Maker FCP, and TestSwarm for iOS. I learned how valuable blogging and communicating with the community is. I blogged and tweeted much more in these past four months than I ever have, and I plan to do more. Making more use of Lighthouse and GitHub this semester also gave me good insight into tools that I can use in future projects that I create. Although this course has come to an end, I still plan to do the work needed in order to land the UndoManager in Butter.

This experience has been great; every programming student should take Dave Humphrey‘s open source classes.

• testing
• putting the code in the right place
• adding more commands
• adding shortcuts to command objects

I have now added 9 tests that make sure that the UndoManager is working correctly. I have also moved the code from butter-main.js, which is now just called main.js, to a new file named src/core/commands.js. For now, all of the commands and convenience functions are in this file. I have added more command objects, and now Butter can undo and redo the following

• adding track events
• removing track events
• adding tracks
• removing tracks
• changing the media url

All of the commits are here, and the diff is here. This is definitely my biggest patch that I have submitted to Butter, which is, so far, 399 additions and 21 deletions. There is still work that needs to be done. For the new code introduced to work cleanly, I still need to create a diagram of how all of this will work. Before I can do that, though, I need a diagram of Butter’s current design. Once the diagram is complete, everyone can see from a high level how objects interact with each other, and we can develop a workflow of what needs to be done when a new command needs to be created.

Implementing undo and redo for adding and removing a track was interesting, because when I deleted the first track and then undid the change, it was added as the last track. The problem ended up being an ordering issue, and I had to change a bit of code in a few areas. In the addTrack function in src/core/media.js, I added another argument, named order. If order is passed in, then that is the value that is set on the track. To actually make the the ordering work, I changed the code in src/timeline/track-container.js and src/timeline/trackhandles.js to use the insertBefore function rather than append on the container element. For example, src/core/trackhandles.js has the following code:

_listElement.insertBefore( trackDiv, _listElement.children[ e.data.order ] );

While testing out the undoing and redoing of adding/removing track events, I noticed a bug. The command object that is responsible for adding a track event was creating a new track event every time it was told to execute. This caused some weird behaviour, which took some digging to find. The fix was simply to create a new track event based on the one that was previously created instead of always creating a fresh one.

There is not enough time for this to be 100% complete before the semester, especially since I need to wait for a diagram to be created, and the new code still needs to go through review. However, I do plan on finishing this, because I would like to see it land in Butter, so that users can work on their projects knowing that they can undo and redo their changes.

I wrote about the progress made in my last release in the blog post, “Butter: Bug 463, the Undo Manager”.

Because this release was focused on architecture, I spent all of my time presenting and discussing the pattern being used, and the possible changes that need to be made. I was also active on GitHub and LightHouse, answering questions and concerns.

As a group, we went through code examples, what the pattern looks like when implemented in Butter, and how this will fit in with Butter’s current architecture. Although there was not much code added this release, there was a lot of discussion so that everyone is on the same page and understands the changes being made.

I also slightly changed the code, and submitted it in the pull request. The new commit introduced a makeCommand factory function, which must be called every time a new command is created. This factory function will automatically register the command with the UndoManager, and returns the command object to the caller, but without the undo function. Not returning the undo function as part of the command object to the caller is to ensure that undoing is only done through the UndoManager.

To summarize, to add a track event to Butter, and have undo and redo automatically done, the code within Butter would call butter.addTrackEvent. Behind the scenes, addTrackEvent will use a command and be registered with the UndoManager. If a button on the UI wants to add a track event, it will simply call execute on the command created by addTrackEventCommand. The key point here is that whether the command is being called from the UI or from Butter, there is only point of entry for the code.

In order to make everything more clear, a diagram needs to be created. I will be creating one as soon as I have a diagram of how Butter currently works. Additionally, the following items still need to be completed before this code lands in Butter

• testing
• putting the code in the right place
• adding more commands
• adding shortcuts to command objects

• Bug 434: Let users move track events together.
• Bug 458: Put the code that validates the times entered for track event updates in a better place.
• Bug 493: Focus the track event editor if a track event is doubled clicked and the editor is already open and in the background.
• Bug 463: Implement undo/redo backend.

I wrote about bug 434, 458, and 493 in this post. I wrote about bug 463 in another post, here.

Bug 434 is still a work in progress. Bobby already started working on implementing our own drag and drop code in src/util/dragndrop.js.

Bug 458 is in review, but I need to fix a problem with my solution. The problem is that when a user hits tab after editing one of the fields, the editor will close. However, there is another bug which is open that will allow users to edit another track event from the track event editor. This means the editor should not close automatically on its own, and thus I am going to suggest leaving the closing of the editor up to the user.

Bug 493 is staged; I ran into this bug while working on bug 458.

Bug 463 is up on GitHub now in order to get feedback so I can put the code in the proper place and adjust the design as necessary.

Currently

• Control-z is undo, and control-shift-z is redo.
• Only track events that are added can be undone and redone.

In src/timeline/media.js, adding a track event, before the UndoManager was implemented looked like this:

      var trackEvent = track.addTrackEvent({
        popcornOptions: {
          start: start,
          end: start + 1,
          target: defaultTarget.elementID
        },
        type: type
      });

With the UndoManager in place, it now looks like this:

      var trackEvent = butter.addTrackEvent({
        track: track,
        type: type,
        popcornOptions: {
          start: start,
          end: start + 1,
          target: defaultTarget.elementID
        }
      });

It remains largely the same, except that behind the scenes, butter.addTrackEvent creates the command object and registers it with the UndoManager. After this is done, control-z can be used to undo, and contol-shift-z can be used to redo the action. The commands are placed on an undo and redo stack as appropriate, and currently there are no restrictions on how many commands can be undone/redone.

Having the UndoManager in place will provide Butter with many benefits. Some of them are

• Creating macro commands.
• A simplified user interface: The UI only needs to know to call the command’s execute function.
• Loosely coupled design.

I will be getting feedback and fixing the design in the coming weeks, and we will, piece by piece, change all undoable actions in Butter to make use of the commands and undo manager.

Bug 434 is allowing users to drag multiple track events together. I talked about the problems I ran in into in this post. Since then, I got the multiple events dragging using an external plugin. For now, the code is here. It doesn’t completely replace the behaviour provided by jQuery’s draggable plugin, though. There is no scrolling, and it cannot yet drop to other tracks. On Lighthouse I have asked if we should be creating our own draggable and droppable code for Butter, since we do not want to add more external dependencies.

In Butter, shift-clicking a track event adds it to the group of selected track events. In the code, we need to determine, after a user clicks on a track event, whether the intended action is to drag, or to deselect all but the currently selected track event. To solve this, I kept track of the amount of time between the mousedown and mouseup event on the trackevent. If the mouseup occurs between 100 and 300 milliseconds after the mousedown event, which is an arbitrary length of time I chose after a little bit of testing, then I assume that the user intended to deselect all but the currently selected elements. Otherwise, I assume the user wanted to drag. The code for that looks like this:

    function onTrackEventMouseDown( e ){
      var start = +new Date();
      e.originalEvent.target.addEventListener( "mouseup", function(){
        var trackEvent = e.trackEvent,
            corn = trackEvent.popcornOptions,
            originalEvent = e.originalEvent,
            end = +new Date(),
            timeElapsed = end - start;

        if ( timeElapsed >= 100 && timeElapsed  1 ){
            trackEvent.selected = false;
          }
          else {
            trackEvent.selected = true;
            if( !originalEvent.shiftKey ){
              for( var t in _tracks ){
                if( _tracks.hasOwnProperty( t ) ){
                  _tracks[ t ].deselectEvents( trackEvent );
                } //if
              } //for
              _selectedEvents = [ trackEvent ];
            }
            else {
              _selectedEvents.push( trackEvent );
            } //if
          } //if
        }
    });

For the actual dragging, I used the following code, which uses a little bit of arithmetic to determine how far each selected track event is allowed to move:

    function dragInit( e, dd ) {
      var selectedEvents = $( "div[selected=true]" ),
          leftPositions = [],
          lowestLeft,
          p = $.event.special.drag.callback.prototype;

      for ( var idx = 0; idx < selectedEvents.length; idx++ )  {
        leftPositions.push( $( selectedEvents[ idx ] ).position().left );
      }

      p.lowestLeft = Math.min.apply( Math, leftPositions );

      return selectedEvents;
    }

    function dragStart( e, dd ) {
      var container = $( _element.parentNode.parentNode );
      dd.limit = container.position();
      dd.limit.left = dd.limit.left + $( this ).position().left - dd.lowestLeft;
      dd.limit.bottom = dd.limit.top + container.outerHeight() - $( this ).outerHeight();
      dd.limit.right = dd.limit.left + container.outerWidth() - $( this ).outerWidth();
    }

    function drag( e, dd ) {
      $( this ).css({
        top: Math.min( dd.limit.bottom, Math.max( dd.limit.top, dd.deltaY ) ),
        left: Math.min( dd.limit.right, Math.max( dd.limit.left, dd.offsetX ) )
      });
    }

    var handles;
    this.activate = function(){
      if( !handles ) {
        $( _element ).css({
          zIndex: 9001
        })
        .drag( "init", dragInit )
        .drag( "start", dragStart )
        .drag( "end", movedCallback )
        .drag( drag, { relative: true } )
        .resizable({
          autoHide: false, 
          containment: "parent", 
          handles: "e, w", 
          scroll: false,
          stop: movedCallback
        });

        handles = _element.querySelectorAll( ".ui-resizable-handle" );
        function toggleHandles( state ){
          handles[ 0 ].style.visibility = state ? "visible" : "hidden";
          handles[ 1 ].style.visibility = state ? "visible" : "hidden";
        } //toggleHandles
        _element.addEventListener( "mouseover", function( e ){
          toggleHandles( true );
        }, false );
        _element.addEventListener( "mouseout", function( e ){
          toggleHandles( false );
        }, false );
        toggleHandles( false );

      } //if

    }; //activate

Bug 458

Bug 458 is putting the code that validates the times entered for track event updates in a better place. Before, this code was only in the editor, which means that if a track event was altered programatically, the validation would be bypassed. I moved the validation from src/editor/editor.js to src/core/trackevent.js; all track event updates have to hit the update method in src/core/trackevent.js, so it is a good place to put the validation (for now, until MVC is implemented).

As discussed with Davide Humphrey, I also made a few optimizations to the code. Namely, I changed calls such as Number( popcornData.start ) to +popcornData.start, and isNan( popcornData.start ) to popcornData.start != popcornData.start (there is, or at least, was, a bug in isNan).

I also changed the logic that decides whether or not to close the editor. Before, there was a boolean variable passed in that would make this decision. I removed this, and just made it so that the editor only closes when the trackeventupdate event is received (because that only fires if the track event was successfully updated). This also gives authors of future editors more control of when to keep the editor open, and when to close it.

The pull request is here.

Bug 493

I filed bug 493 when I was working on bug 458. If the track event was double clicked and the editor was already open, but behind the main browser’s window, then the editor did not come to the front. It stayed hidden in the background, which makes it seems like the double click didn’t work. I changed it so that the track event editor’s window comes to focus if it already open; the pull request for that is here.

The first two bugs are from Butter, bugs 256 and 390. As explained in the post, I did the work needed for the two tickets, but the code that was actually staged came from Bobby, who used the work from tickets 256 and 390, and merged it with bug 313, and I did the peer review.

I also worked on bug 434, which ended up being a lot more work than I expected. Even though I have some code working, this bug will need a lot more put into it before it’s done. Since the jQuery UI draggable plugin will no longer be used, a lot of functionality that used to come for free now has to be implemented by us. Before getting too far with my solution, David Seifried told me that I should talk to Bobby, since he talked about writing our own drag-and-drop functionality in order to reduce the dependencies in Butter.

In addition, I fixed up bug 459. Even though it was easy, I hope that having a more meaningful and readable error message will add to the user experience.

Finally, I worked on getting TestSwarm for Popcorn on iOS to work. Now TestSwarm can run Popcorn tests on the iOS simulator and the physical device!

The reason we need to create our own custom app is because Safari requires user action in order for media playback to occur. This doesn’t work for the purposes of automated testing. When first thinking about this problem, I initially thought that I was going to need to inject JavaScript into the WebView that would eventually lead to the Objective-C triggering the video to play. But, the fix for this actually ended up being simple, and I found it when I wrote an older post, DPS911 Release 3. As stated in that post, the UIWebView class has a mediaPlaybackRequiresUserAction property that I can set to no, which will let videos play automatically.

The code for that is:

[[self theWebView] setMediaPlaybackRequiresUserAction:NO];

And it works! Just one line. Now TestSwarm can run Popcorn tests on iOS using the simulator and the physical device!

Before, when an invalid start or end time for a track event was entered, the error message displayed was:

You’ve entered an invalid start or end time. Please verify that they are both greater than 0, the end time is equal to or less than the media’s duration, and that the start time is less than the end time.

I have changed it to

You have entered an invalid in or out time. Please verify that

• Both in and out times are greater than 0.
• The out time is equal to, or less than the media’s duration (duration of the media, in seconds, displayed here).
• The in time is less than the out time.

Three things to note are:

• The error message now refers to start and end times as in and out times, since that is how it is displayed to the user in the editor.
• The media’s duration is displayed in the error message.
• The error message is now displayed as a bulleted list for readability.

The pull request for this patch is here.

Initially, I was thinking that since the dragging behaviour already exists in the track event, I could just have a loop that goes through all of the selected track events, and trigger the dragging behaviour on each one. After trying various things in the code, I realized that making this trigger happen wasn’t obvious.

I started doing some searches, and ended up reading this post on StackOverflow. I implemented something similar to the selected answer, which is to implement the helper function in jQuery’s draggable to return all of the elements to be dragged appended to one container. The suggestion in the post was this:

$('#dragSource li').draggable({
  helper: function(){
    var selected = $('#dragSource input:checked').parents('li');
    if (selected.length === 0) {
      selected = $(this);
    }
    var container = $('
').attr('id', 'draggingContainer');
    container.append(selected.clone());
    return container; 
  }
});

This person also put up a demo on JSBin, and it seemed like this is exactly what I wanted. But, after trying it in Butter, it ended up having unsuitable behaviour. Everything that was appended to the div appeared on top of each other, and that means that the track events were losing their place when they were being dragged. I could have done some styling with the container div so that the track events would keep their position, but this seemed like a hack that wouldn’t work.

Moving on, I found this post on the jQuery UI Google Group, which says to capture the mousedown event on the container, and then from there trigger the even on the draggables, which should start the dragging operations on all of the events. The code for that looks like this:

$(".draggable").draggable();

$("#container").mousedown(function(e) {
    $(".draggable").trigger(e);
});
​

This works for the first draggable element in the HTML, but not for the rest. There is also an error thrown in the console (I deleted much of error here because it’s too long):

jQuery.event.dispatch
jQuery.event.add.elemData.handle.eventHandle
jQuery.event.trigger
jQuery.fn.extend.trigger
jQuery.extend.each
jQuery.fn.jQuery.each
jQuery.fn.extend.trigger
(anonymous function)
jQuery.event.dispatch
jQuery.event.add.elemData.handle.eventHandle
jQuery.event.trigger
jQuery.fn.extend.trigger
jQuery.extend.eachjquery-1.7.1.js:658

.....

jQuery.event.trigger
jQuery.fn.extend.trigger
jQuery.extend.each
jQuery.fn.jQuery.each
jQuery.fn.extend.trigger
(anonymous function)

Changing the code up a bit would just give me another error, saying that there is too much recursion. So I tried to do this:

$(".draggable").draggable();

$("#container").mousedown(function(e) {
    $(".draggable").trigger(e);
});

$(".draggable").mousedown(function(e) {
    e.stopPropagation();
});

This does not throw an error, but again there is the issue that the dragging operation only works on the first draggable in the HTML.

Similarly, I found this bug filed in jQuery three years ago. It suggested writing the code like this:

$('#draggable').one('mousedown', function(e) {
    $("#draggable").draggable().trigger(e);
});

This didn’t work either, since the mousedown event was only captured once.

I then found another discussion on the jQuery forum, where rdworth mentions that jQuery uses the simulate plugin, and linked to examples of how it’s used in the jQuery unit tests. This looked like it was a bit overboard for what we’re trying to do, so I needed to find something else.

I was trying to stay away from introducing another external plugin to Butter, but it seems that I need to. I found this multidraggable jQuery UI plugin. I tried it out, and it works, but the one problem is that it doesn’t let the developer choose which elements are to be dragged. The way the elements are selected to be dragged is done within the plugin itself, using a control-click. This is an issue because Butter has it’s own way of deciding which track events are selected.

Finally, I found another plugin by ThreeDubMedia, which provides both drag and drop plugins. These plugins are set up to give the developer a lot of control, which is good for our case. I now have some code that moves multiple track events, but now that the jQuery UI draggable plugin is not being used, other areas need to be rewritten. For example, the scrolling feature that’s available automatically with the jQuery UI draggable plugin doesn’t seem to exist in the plugin provided by ThreeDubMedia, which means code for that has to be created.

So far, this is what I have:

function drag ( ev, dd ) {
    $( this ).css({
        top: dd.offsetY,
        left: dd.offsetX
    });
}

$( _element ).css({
    zIndex: 9001
});
$( _element ).drag( "init", function() {
    return $( "div[selected=true]" );
}).drag( "end", movedCallback ).drag(drag, {
    relative: true
}).resizable({ 
    autoHide: false, 
    containment: "parent", 
    handles: "e, w", 
    scroll: false,
    stop: movedCallback
});

Oh, and I also found an accidental global variable, _selectedEvents, in src/timeline/media.js. I will also fix that in the patch I submit.